But Who Would Insure a Criminal?
This is the first thing people say, and the version they say is usually not the strong version.
The weak version is why would any company want this business? That one answers itself: because it is business. Insurers write cover for skydiving, offshore drilling, war zones, and the personal liability of people whose entire job is making consequential decisions badly. Risk is not an obstacle to the industry, it is the product. If the risk can be estimated and the premium exceeds the expected loss, someone will write it.
The strong version is a legal one, and it took me longer than it should have to take it seriously.
The objection, properly stated
In most jurisdictions you cannot insure yourself against your own intentional wrongdoing. It is not that no insurer will sell it; it is that the contract is void as against public policy, on the reasoning that a policy which pays out for deliberate misconduct is a subsidy for deliberate misconduct.
That rule is correct and I would not want it removed. But it appears to gut the proposal on this site, because the whole point of an extinction bounty is to attach a large penalty to conduct we want deterred — and if the penalty cannot be insured, we are back to the judgment-proof problem with no way out of it.
The category error
The objection assumes the thing being insured is deciding to endanger the world. It is not.
What is being insured is crossing a defined statutory line. And the realistic population of people who cross a defined statutory line in frontier AI work is overwhelmingly not composed of people who meant to. It is composed of people whose FLOP accounting differed from the regulator’s. People who inherited a training pipeline and did not audit what was in it. People who ran the eval, got a result they misread, and shipped. People whose lawyer told them the carve-out applied. People who were three levels down an org chart and did what the org chart said.
This is what compliance violation looks like everywhere else it exists, and there is no reason to expect AI to be the exception. The deliberate world-ender is a rounding error in the population and the least likely to be deterred by anything. The mechanism is aimed at the enormous middle: people who would have stopped if stopping had been the cheaper option, and for whom the premium is what makes it cheaper.
So the public policy rule and the mechanism are not actually in conflict. The rule bars insuring intentional harm. Almost nothing here is intentional harm.
The industry already solved this
The interesting part is that this problem — cover someone against serious allegations, but do not end up subsidising deliberate wrongdoing — is not hypothetical. It is the central design problem of directors and officers insurance, and the industry’s answer is a piece of contract drafting worth stealing wholesale.
D&O policies carry conduct exclusions for deliberate fraud, dishonesty, and willful statutory violation. But the better-drafted ones are triggered only on final adjudication : the exclusion does not bite on an allegation, or an indictment, or even a first-instance verdict. It bites when a court has finally determined that the conduct was deliberate, in some policies only after appeals are exhausted. Until then the policy funds the defense, and where the exclusion ultimately does apply the insurer carries the burden of clawing the money back.
Transplant that and the bounty policy practically writes itself. You are covered for crossing the line. You are not covered if a court finally determines you crossed it on purpose. In the interim you are defended.
That last clause does more work than it looks like. It means the insurer’s money is on the same side as the accused during the proceeding, which is an adversarial check on exactly the failure mode page 12 worries about — the false or opportunistic claim. A bounty hunter bringing a weak case is not up against a frightened researcher with a public defender. They are up against an insurer with a financial interest in establishing that nothing deliberate happened, and the resources to do it.
Other things that already exist
Once you look, the shape is everywhere:
- Professional indemnity and errors & omissions cover negligent acts, errors, and omissions by people whose work is consequential and technical. That is the target population almost exactly.
- Environmental liability covers statutory cleanup obligations — a government-defined line, a large number attached to crossing it, and a functioning insurance market around it.
- Surety bonds are closer still in structure: a third party guarantees the principal’s obligation, pays if the principal defaults, and retains recourse against the principal afterwards. The state gets made whole; the wrongdoer still owes. This is the answer to anyone who thinks insurance means the guilty party walks away costless.
- Price-Anderson, the mandatory pooled catastrophe cover that has underwritten American civil nuclear power since 1957, which the nuclear-precedent paper treats as the working template.
None of these markets collapsed. None of them turned out to be a licence to offend. Several of them are the reason the underlying activity is possible at all.
The part that is actually an advantage
Here is the thing I did not appreciate until I wrote this note out.
The liability-insurance proposals this site compares itself to try to insure against the harm. That is genuinely hard, and the honest ones admit it: the catastrophic tail is uninsurable by construction, because no premium covers extinction and no insurer survives paying for it. You end up needing government backstops, pooled layers, and a frank acknowledgement that the top of the distribution is uncovered.
An extinction bounty is not the harm. It is a defined statutory number — call it $2.25 million — attached to a defined act. It is bounded, it is knowable in advance, and it does not scale with how bad the outcome was. That makes it ordinary from an underwriting perspective. It is a much smaller and better-behaved object than the thing it is standing in for.
The bounty is insurable precisely because it is not the damage. That is not a concession, it is the design working.
Where this gets harder
Three honest problems.
There is no claims history. D&O is priced off decades of data. Nobody has a loss history for frontier AI bounty claims because there have been none. Early premiums would be guesses wrapped in a loading, and the first few years would be priced badly in some direction nobody can predict.
Willful exclusion reopens a door. If the policy excludes deliberate violations, then in the deliberate cases the insurer owes nothing and could in principle claim the bounty itself. I have written that up separately ; the short version is that it needs a rule against insurers claiming on their own insureds, and that the rule is cheap.
Correlated risk. Insurers can absorb uncorrelated losses. If a statutory line turns out to have been crossed by every lab simultaneously — because the line was ambiguous and everyone read it the same wrong way — that is a single event with many claims, which is the shape that ruins carriers. This is an argument for pooling, for reinsurance , and possibly for the government backstop that Price-Anderson provides. It is not an argument against the mechanism, but it is the reason the mechanism cannot be built out of nothing but private contracts.