The Two-Minute Pitch
The whole mechanism, as fast as it can be told.
The dangers I worry about most do not respect borders. Frontier AI, engineered pandemics, and whatever comes after them can in principle be built by a small number of people, anywhere, using equipment nobody has to smuggle. Our enforcement machinery is still national. Treaties are slow, international law is weak, and policing a cross-border actor is close to impossible. Meanwhile the obvious remedy — simply stopping the research — would cost us most of what makes modern civilisation worth defending.
Extinction bounties are my attempt at a way through. Rather than build an agency and hope it can find offenders, price the offense and let everyone else find them.
Three moving parts
A statutory penalty attaches to a narrow, defined class of activity a society has decided it wants slowed down — training a frontier model above some threshold, say, or gain-of-function work on a pathogen with pandemic potential. The number is large, and it is set in public by a legislature that has to own it.
A bounty goes to whoever provides evidence leading to a successful prosecution. Anyone, anywhere, of any nationality. The money comes out of the penalty itself, not from a budget — which is precisely how the SEC’s whistleblower program already works, and it has worked well enough to pay out over a billion dollars without a line item.
Private liability insurance, held by the individual researcher rather than the firm, covers the penalty. This is the part that does the real work, and the part most people skip. A penalty large enough to deter is a penalty larger than almost any individual can pay, which normally makes it empty — the judgment-proof problem, and it bites unusually hard here, since frontier researchers tend to be young, well-paid and not yet wealthy. Insurance converts an uncollectible fine into a monthly premium somebody will actually feel. Doctors already live this way.
Why it has to be the individual
Because a premium is a price on your colleagues’ silence.
If the firm carries the cover, the firm has bought everyone’s silence in a single transaction, and the people inside it have no particular reason to break ranks. If each researcher carries their own, every one of them is separately exposed, separately underwritten, and separately able to walk into their own insurer’s office and get paid for what they know. Firm cover pools the conspiracy. Individual cover divides it. That asymmetry is the whole mechanism, and page 08 is where I argue it properly.
What it buys
Incentives that point the same way. Insurers monitor clients to avoid payouts. Bounty hunters compete to find violations. Researchers notice that a job which does not risk ending the world still pays extremely well.
Self-funding enforcement. The penalties pay for the enforcement. No appropriation, no agency budget to cut, no taxpayer to persuade — which, as page 09 argues, is most of why nobody has tried paying for this kind of deterrence before.
A low bar for international adoption. No new treaty body, no harmonised penal code — a number and a court, which most jurisdictions already have. That is page 10 , including the uncomfortable part: if this is legitimate when my country does it, it is legitimate when yours does it to me.
Nothing to capture. There is no licensing board with discretion, because there is no licensing board. Page 11 sets this against the alternatives.
What success looks like
Not prosecutions. If this works, almost nothing happens: a handful of research programmes quietly never start, a few people take the other job offer, and there is nothing to report. The chilling effect is the product , not a side effect I am apologising for. It also means the mechanism is hard to evaluate by the usual metrics, which is a real cost and one I would rather state than bury.
Nothing here is specific to AI. The same shape applies to any frontier activity a society wants to slow without banning outright; AI is simply what worries me most at the moment.
What this is not
This is a mechanism-design argument published for debate. It is not legal advice, not a call to action, and not aimed at anyone — see non-targeting . It is also not original in its general form: Robin Hanson proposed fine-insured bounties as a wholesale replacement for criminal law, and I have been arguing the narrow existential-risk version of it since 2022 . I reserve extinction bounties for that narrow version.
The objections are real, and I have tried to give them their due in page 12 rather than here. If you want the argument rather than the summary, start at the sequence .