Should X-Risk Liability Insurance Be Voluntary or Mandatory?
This is the design fork with the largest downstream consequences, and when I first wrote about it I concluded there were no knockdown arguments either way. I have changed my mind — not about which of the two options wins, but about the question being a choice between two options.
The case for mandatory
Hanson’s original version is unambiguous. His proposal is to require that everyone get insurance “to cover a high maximum liability for legal violations. At least a few million dollars,” and the enforcement structure follows from it: “Only the crime of not obtaining full legal liability insurance might need an official non-fine punishment.”
That is elegant. The entire apparatus of criminal punishment collapses into a single offense — being uninsured — and everything else becomes a bill. It also solves the problem that page 08 spends most of its length on: if coverage is compulsory, the judgment-proof defendant stops existing. There is always someone solvent standing behind the fine.
The precedents are real. Most American states require auto liability insurance. Price-Anderson has required nuclear operators to carry cover since 1957. Neither is regarded as exotic.
The case against is not the one people expect
The usual objection to mandatory insurance is that it is heavy-handed for a mechanism that will touch a few thousand people. That is true but weak — the mandate would be narrow, the population is well-paid, and premiums for people doing ordinary non-frontier work would round to nothing.
The real objection is that mandatory insurance is a licensing regime wearing a disguise.
Work it through. If you cannot do frontier AI research without coverage, and insurers may decline to cover you, then insurers decide who may do frontier AI research. Not the legislature, not a regulator with a statutory mandate and an appeals process — a private underwriting committee, applying criteria it does not publish, with no obligation to explain a refusal and no route to challenge one.
The entire appeal of this mechanism over conventional regulation was that it does not require an authority sitting in judgment on who is permitted to do the work. Make the insurance compulsory and you have reinstated exactly that authority, privatised it, and stripped out the due process that made the public version tolerable. That is not a smaller government footprint. It is the same footprint with worse accountability.
Hanson, to his credit, does not hide from where this goes. Asked what happens to someone whose record makes them uninsurable, he suggests they “might only be able to afford premiums if they agreed to stay on a gated and isolated work farm.” He means it as a live option to be priced rather than a dystopia, and within his framework it is coherent. I think it is also a fairly direct demonstration that once insurance is the gate, the insurer’s terms become the terms of your life.
The case for voluntary, and why it fails on its own
Voluntary has obvious appeal. The market discovers the price with nobody mandating anything, no new bureaucracy appears, and people who decline coverage simply carry their own risk.
The problem is who declines. Insurance bought voluntarily is bought by the conscientious — people who think carefully about tail risk, which correlates almost perfectly with being the people you were least worried about. The person who shrugs at a $2.25 million exposure because they have no intention of thinking about it is precisely the person the mechanism exists to reach, and they are the first to go bare.
So pure voluntary selects against itself. You end up insuring the careful and leaving the careless uncovered and judgment proof, which is the situation you started in.
The third option
Voluntary, but with the uninsured position made genuinely unattractive: coverage is not required, and going without it means the liability is uncapped, personal, and non-dischargeable in bankruptcy.
You may work uninsured. Nobody stops you, no licence is refused, no private committee passes judgment on your career. But you are then personally on the hook for the full bounty, it does not go away when you file, and it follows you. Almost anyone with assets, future earnings, or a mortgage will buy the policy, and they will buy it because the alternative is worse rather than because the state told them to.
This keeps the property I care about most: an insurer declining to cover you is a price signal, not a prohibition. You can still do the work. You just carry the risk yourself, visibly, which is information for everyone around you — including the lab deciding whether to employ someone who could not get covered.
It also keeps the state out of the business of deciding who may think about what, which is the line I do not want this mechanism to cross even when crossing it would be effective.
The tension I have to admit
There is a hole in that, and it is the same hole as everywhere else.
Uncapped non-dischargeable liability does not deter a judgment-proof person. That is the entire argument of page 08 — above the ruin threshold the schedule is flat, and making it more flat by adding non-dischargeability changes nothing for someone who has nothing. So the third option works on everyone with something to lose and does no work at all on the people it most needs to reach.
I do not have a clean answer. The partial answer is that the population this targets is the one described on page 08 — young, liquid-poor, but with enormous future earnings — and non-dischargeability bites future income specifically, which is the one asset they definitely have. A twenty-eight-year-old facing a non-dischargeable seven-figure liability that follows them for forty years is not judgment proof in the way a genuinely destitute defendant is. They are illiquid, which is a different condition with a different remedy, and the remedy is the policy.
For the residual — people with no assets and no prospects who nonetheless end up in frontier AI work, which I think is a very small set — nothing in this mechanism works, and I would rather say so than pretend otherwise. That is what the note on truly broke offenders is for.
Where I land
Voluntary with teeth. Not because the mandatory case is weak — it is strong, and Hanson is right that it makes the system cleaner — but because the cleanliness is purchased with a power I do not want anyone to have, least of all a party with no obligation to explain itself.
I hold this more loosely than most things on this site. If someone showed me that voluntary-with-teeth leaves enough of the population uncovered to collapse the arithmetic , I would take the mandate and the licensing regime that comes with it, and be unhappy about it.