Extinction Bounties

Policy-based deterrence for the 21st century.

Glossary

This argument borrows vocabulary from insurance, law and economics, and I would rather define the words than assume them. Terms marked in the text like qui tam show their definition on hover, and clicking one brings you here.

Nothing below is a term of art I invented. Where a definition is contested I have given the version this site uses rather than pretending there is consensus.

adverse selection

When the people most eager to buy insurance are the people most likely to claim.

The reason voluntary cover tends to be bought by the careful. It is also why refusing to be monitored is itself information: declining a discount in exchange for telemetry tells the underwriter something.

agglomeration

The productivity gain from firms and workers clustering in one place.

Why frontier labs sit in the most expensive metros on earth rather than the most permissive jurisdictions, and the main reason the relocation objection is weaker than it sounds.

ambiguity aversion

Charging extra not for known risk but for not knowing the probability.

Kunreuther, Hogarth and Meszaros found insurers strongly averse to imprecise probabilities. The professional asymmetry explains it: overestimating a risk loses you an account, underestimating it produces a visible loss ratio.

blue wall of silence

The refusal of enforcers to report misconduct by other enforcers.

The problem Hanson's design attacks by letting anyone claim a bounty on anyone, including on other bounty hunters. It is the reason competition among enforcers is more corrosive to silence than any oversight body.

Brussels effect

Anu Bradford's term for the EU setting global standards without a treaty, via market access.

Firms rebuild products to reach the EU market, find that running two versions costs more than one, and export the stricter rule everywhere. Then they lobby their own governments to impose it domestically so competitors face it too.

claims-made

A policy that responds to claims made during the period, rather than to events occurring in it.

Claims-made policies typically oblige the insured to notify circumstances that might give rise to a claim. That notification duty is what would let a premium reprice on a quiet settlement rather than only on an adjudicated one.

combined ratio

Claims plus expenses divided by premiums. Below 100% means underwriting at a profit.

The number that decides whether an insurer's float is free. Buffett's point is that cost-free float is not the industry norm - most years, premiums have been inadequate to cover claims plus expenses.

deferred prosecution agreement

A settlement suspending prosecution in exchange for conditions, without a conviction.

How the Meng Wanzhou matter ended. Worth knowing because it is the ordinary way large enforcement disputes actually terminate, rather than by trial.

D&O insurance

Directors and officers liability cover: protects individuals against claims arising from their corporate roles.

The closest existing analogue for insuring a person rather than a firm against consequential decisions, and the source of the final-adjudication drafting this proposal would borrow.

dual criminality

The requirement that conduct be an offence in both countries before extradition is available.

A requirement in most extradition treaties, and the reason a bounty regime cannot expect to reach people in non-adopting states by asking for them. It is also why the design leans on arbitral awards, which travel much better than defendants.

experience rating

Pricing a policy on the insured's own claims history rather than only on the class average.

The mechanism by which insurance restores the incentive that moral hazard removes. It is also why an individually rated bounty policy survives an employer reimbursing the premium: the lab pays more for the riskier researcher either way.

FATCA

The US statute requiring foreign financial institutions to report accounts held by US persons.

Enforced by withholding on US-source payments, which is a lever over private institutions rather than over governments. The clearest demonstration that financially mediated rules travel across borders better than almost anything else states do.

fee-shifting

Making the losing party pay the winner's costs.

The ordinary tool for suppressing speculative litigation, and the main dial available for tuning the false-positive rate of a bounty regime down to whatever level is wanted.

final adjudication

A conduct exclusion that bites only once a court has conclusively found the conduct deliberate.

Standard in better-drafted D&O policies, sometimes requiring appeals to be exhausted. Until then the policy funds the defence. It is how an industry insures people against serious allegations without subsidising deliberate wrongdoing.

float

Money an insurer holds but does not own: premiums received before claims are paid.

Because claims on long-tail lines are settled years after the premium arrives, the insurer invests the difference in the meantime. Where underwriting breaks even or better, the cost of that money is zero or negative - the insurer is paid to hold it.

Goodhart's law

When a measure becomes a target, it ceases to be a good measure.

Goodhart's 1975 observation, in Marilyn Strathern's compression. The reason a licensing regime's written safety criteria stop describing safety the moment they become an exam that capable people are motivated to pass.

hold-up problem

Where one party's sunk investment lets the other renegotiate terms in their favour afterwards.

Why buying silence is a subscription rather than a purchase: each payment increases what the payer stands to lose, which increases what the blackmailer can demand next time.

informant economy

A standing arrangement in which reporting on others is a paid activity.

The thing this proposal builds, and the objection it is least able to answer. The False Claims Act is the reassuring comparison; twentieth-century secret police the alarming one, and the difference is how broad and how well-defined the covered class is.

instrumental convergence

That a wide range of final goals imply the same intermediate goals: acquire resources, preserve yourself, keep your options open.

Omohundro's basic drives. The reason a system need not be hostile to be dangerous - it only needs a goal and enough capability to pursue it.

judgment proof

Unable to pay what you have been found liable for, so the penalty above your net worth does no work.

Steven Shavell's 1986 term. The important part is not that such a defendant underpays afterwards but that they take too little care beforehand, because harm exceeding their assets never enters the decision. This is the problem the insurance leg exists to solve.

lex mercatoria

The medieval law merchant: privately adjudicated commercial law operating across jurisdictions.

It existed because traders crossing borders needed adjudication no single sovereign could supply. Modern international arbitration is its descendant, which makes private cross-border dispute resolution one of the oldest working examples of polycentric law rather than a novelty.

loading

The margin an insurer adds on top of expected loss, covering expenses, cost of capital and profit.

The uninteresting half of a premium. The interesting half is the expected payout, which is where all the information about your conduct lives.

moral hazard

Insurance reducing the insured's incentive to avoid the loss.

The standard objection to insuring anything. It is answered here by experience rating: the premium tracks your conduct continuously, so the incentive is restored at the margin rather than removed.

Nash equilibrium

A state where no player can improve their position by unilaterally changing strategy.

The thing this site is trying to move. The claim is that shifting the equilibrium of whether to take the job at all is easier than solving alignment.

near-miss liability

Scaling damages to the risk created rather than to the harm that happened to occur.

Gabriel Weil's central innovation, and a genuinely clever solution to the problem that the harms a court can litigate are never the harms we care about.

New York Convention

The 1958 treaty obliging 172 states to recognise and enforce each other's arbitral awards.

A wider and more reliable enforcement network than exists for ordinary court judgments. Article V(2)(b) lets a state refuse enforcement contrary to its own public policy, which is the escape hatch any unwilling jurisdiction would use.

orthogonality thesis

That an agent's intelligence and its final goals are independent: any level of capability can accompany any goal.

Bostrom's formulation. It is what blocks the comfortable assumption that something clever enough will arrive at good values on its way up.

overbreadth

A doctrine striking down laws that sweep in protected activity alongside the conduct aimed at.

The companion to vagueness, and the precise risk run by a frontier-AI prohibition broad enough to catch interpretability and evaluation work.

PFIC

A passive foreign investment company: a non-US pooled fund, punitively taxed for US persons.

The other half of the vice. Citizenship-based taxation means a European fund that is unremarkable for a European neighbour triggers a punitive regime and an annual return for an American living next door.

polycentric law

Legal order supplied by several overlapping authorities rather than one territorial monopoly.

Descriptively, it is what a dual national already lives under. The anarcho-capitalist version is voluntary, which is where this proposal parts company with the literature - nobody opts into a bounty statute.

Price-Anderson

The 1957 US statute requiring nuclear operators to carry pooled liability cover, with a government backstop above it.

The working precedent for mandatory catastrophe insurance in an industry whose worst case is uninsurable. Cited by nearly everyone proposing liability insurance for AI.

PRIIPs

The EU regulation requiring a standardised Key Information Document for packaged retail investments.

US funds do not produce one, so an EU retail investor cannot buy a US-domiciled ETF. Half of the vice that catches American citizens living in Europe.

punitive damages

Damages awarded to punish and deter rather than to compensate a loss.

The lever Weil uses to reach uninsurable catastrophe: award punitive damages when an ordinary compensable harm turns out to have been a near miss of something far worse.

qui tam

A suit brought by a private person on the government's behalf, who keeps a share of what is recovered.

From the Latin tag meaning "who sues on behalf of the king as well as for himself." Under the US False Claims Act a private relator files on the government's behalf and keeps between 15% and 30% of the recovery. It is the closest thing in current law to the mechanism this site proposes, and the reason a bounty need not be characterised as a penal claim.

regulatory capture

An agency coming to serve the industry it regulates.

Stigler's account adds the sharper version: the licence becomes an asset, compliance costs fall hardest on entrants, and the regime entrenches incumbents while looking from inside like it is working.

reinsurance

Insurance bought by insurers, spreading large or correlated risks across the market.

The mechanism that lets a single carrier write exposures larger than its own balance sheet, and part of the answer to correlated risk - the scenario where an ambiguous statutory line is crossed by every lab at once.

relator

The private person who brings a qui tam action and takes a share of the recovery.

There is an entire plaintiff-side bar built on relator work, which is the existing profession a bounty-hunting industry would most resemble.

rules versus standards

Whether conduct is specified in advance by a rule or judged afterwards against a standard.

Kaplow's framing: rules for homogeneous frequent conduct, standards for heterogeneous rare conduct. Frontier AI is the textbook case for a standard, which is also why it resists a clean statutory line.

strict liability

Liability without proof of fault or negligence.

Weil's proposal would treat training frontier systems as an abnormally dangerous activity attracting strict liability. It removes the need to prove a developer was careless, which matters when nobody can say what care would have looked like.

surety bond

A third party guarantees an obligation, pays if the principal defaults, and retains recourse against them.

Closer in structure to a bounty policy than ordinary liability cover: the state is made whole immediately and the wrongdoer still owes. The answer to anyone who thinks insurance lets the guilty walk away costless.

tail risk

The low-probability, high-severity end of a loss distribution.

Insurable when bounded, as with hurricanes. Uninsurable when the worst case exceeds any premium anyone could pay, which is the case for the harm this site is concerned with - and the reason the policy covers a statutory bounty instead.

foot-voting

Choosing a jurisdiction by moving to it, rather than by voting within it.

The mechanism behind the relocation objection. It works on capital far better than on people, which is why the elasticity of the target matters so much to whether a rule travels.

utmost good faith

The duty to disclose material facts to an insurer, with nondisclosure grounds for voiding the policy.

Uberrima fides. It is why an insurance relationship can extract information that no regulator could compel, and the stick behind the disclosure channel described on page 08.

underwriting profit

Premiums collected exceeding claims and expenses paid.

The condition that makes float free. Where an insurer underwrites at a loss, the float has a positive cost and the investment income is merely offsetting it.

unraveling result

Where disclosure is cheap, silence is read as the worst case, so the best of the silent disclose - and it iterates.

Milgrom and Grossman, both 1981. It is why an underwriter gets told things nobody compelled anyone to say.

void for vagueness

A doctrine striking down laws too unclear for an ordinary person to know what is prohibited.

The reason deliberate statutory ambiguity is a liability rather than a feature, however useful the resulting caution might be.

Policy-research disclaimer

Extinction Bounties publishes theoretical economic and legal mechanisms intended to stimulate scholarly and public debate on catastrophic-risk governance. The site offers policy analysis and advocacy only in the sense of outlining possible legislative or contractual frameworks.

No legal or financial advice

Nothing here should be treated as a substitute for qualified legal counsel, financial due diligence, or regulatory guidance. Readers remain responsible for ensuring their actions comply with the laws and professional standards of their own jurisdictions.

Exploratory and personal views

All scenarios, numerical examples and opinions are research hypotheses presented by the author in a personal capacity. They do not represent the views of the author's employer, funding bodies, or any governmental authority.

Implementation caveats

Any real-world adoption of these ideas would require democratic deliberation, statutory authority, and robust safeguards against misuse. References to enforcement, penalties, or "bounties" are illustrative models, not instructions or invitations to engage in private policing or unlawful conduct. Nothing here is directed at any identifiable individual — see non-targeting.

No warranty and limited liability

Content is provided "as is" without warranty of completeness or accuracy; the author disclaims liability for losses arising from reliance on this material.

By continuing beyond this notice you acknowledge that you have read, understood, and accepted these conditions.