Glossary

In US tort law, an activity so risky that whoever does it is liable for harm even if they took every care.

Blasting and storing explosives are the classic examples; the modern test is in the Restatement (Second) of Torts. Gabriel Weil proposes classing frontier training this way, descended from Rylands v Fletcher in 1868.

The view that technological and capitalist processes should be sped up rather than restrained.

Its theoretical root is Nick Land, for whom capital and intelligence are one process that owes nothing to human values. Its cheerful Silicon Valley descendant, effective accelerationism or e/acc, is less interested in the philosophy and more in the building.

A chip specialised for the arithmetic machine learning needs: a GPU or similar.

They come by the pallet, cost more than cars, and leave a paper trail through customs, procurement and electricity. That trail is most of what an outside investigator has to work with.

The statistician an insurer employs to work out what losses to expect.

An actuary does not tell you what will happen to one policyholder; they tell you what happens on average to a thousand of them. That distinction is why an insurer can price a risk nobody can observe directly, and why their read is never a verdict about you.

When the people most eager to buy insurance are the people most likely to claim.

The reason voluntary cover tends to be bought by the careful. It is also why refusing to be monitored is itself information: declining a discount in exchange for telemetry tells the underwriter something.

The productivity gain from firms and workers clustering in one place.

Why frontier labs sit in the most expensive metros on earth rather than the most permissive jurisdictions, and the main reason the relocation objection is weaker than it sounds.

The problem of getting a capable AI system to pursue what its designers actually intended.

Unsolved, and this site assumes it stays unsolved for the relevant period. The proposal here is explicitly not an alignment strategy: it is an attempt to move the [[nash-equilibrium]] of whether the dangerous work gets done at all.

Charging extra not for known risk but for not knowing the probability.

Kunreuther, Hogarth and Meszaros found insurers strongly averse to imprecise probabilities. The professional asymmetry explains it: overestimating a risk loses you an account, underestimating it produces a visible loss ratio.

The view that bringing new people into existence is wrong, or always a harm to them.

David Benatar's version rests on an asymmetry: the absence of pain is good even with nobody to enjoy it, while the absence of pleasure is bad only if someone is deprived. On that account never existing always beats existing.

The open preprint server where most machine-learning research is posted, before and often instead of peer review.

Publishing on arXiv is the default in this field, which makes not publishing a signal. A team that stopped posting eighteen months ago has told you something without meaning to.

A system more capable than humanity at the things that decide outcomes.

The thing the [[statutory-threshold|threshold]] is drawn to keep anyone from reaching by accident. Whether the run you are sitting inside in Act I is one is exactly what nobody has checked, and the game's point is that the man-hours accumulate at the same rate either way: the run does not wait for the question to be settled.

A shared record of which organisations have submitted to an outside audit, and which have not.

The value is in the negative entries. A [[third-party-audit]] is voluntary everywhere, so the useful fact is not that a team passed one but that it declined one, and a shared list is what makes declining legible.

The refusal of enforcers to report misconduct by other enforcers.

The problem Hanson's design attacks by letting anyone claim a bounty on anyone, including on other bounty hunters. It is the reason competition among enforcers is more corrosive to silence than any oversight body.

The whole set of policies an insurer currently has [[in-force]].

Insurers think in books rather than policies, because that is the level at which the arithmetic works and the level at which the firm lives or dies. A book with two policies on the same training run is a book with one risk in it, twice.

A lawyer who specialises in bringing bounty claims, usually for a share of the award.

The profession that would appear within months of such a statute, on the model of the existing [[relator|relator's]] bar. Their real product is not advocacy but triage: knowing which cases a court will believe, which is exactly what a prospective claimant cannot judge alone.

An intermediary who places business with insurers on a client's behalf.

A broker's block is never a random sample of the market. People who let somebody else do the shopping are disproportionately the people who did not want to answer questions themselves, which is why buying business wholesale is buying the worse half of it.

Anu Bradford's term for the EU setting global standards without a treaty, via market access.

Firms rebuild products to reach the EU market, find that running two versions costs more than one, and export the stricter rule everywhere. Then they lobby their own governments to impose it domestically so competitors face it too.

The money standing behind an insurer's promises, over and above what it expects to pay.

Capital is what makes cover credible: an insurer with none is selling a promise it cannot keep. It is also what shareholders want distributed, which is the tension every underwriting desk sits inside.

To pass part of a risk you have written to somebody else, for part of the premium.

The verb of [[reinsurance]]. An insurer cedes the top of a loss so that no single claim can end the firm, and keeps the frequent small stuff where its own judgment is worth something.

Lawful behaviour deterred because people fear a law might reach it.

In law, usually a criticism of vague or overbroad rules. This site argues that near the line it is the intended output, while conceding the cost it imposes on people who were never near it.

A demand on a policy: the event the insurer promised to pay for has happened.

Everything an insurer does is a bet about claims it has not seen yet. Under this proposal the claim is the whole fine at once, arriving the month somebody files, which is why a single report can take down every policy written on the same run.

A policy that responds to claims made during the period, rather than to events occurring in it.

Claims-made policies typically oblige the insured to notify circumstances that might give rise to a claim. That notification duty is what would let a premium reprice on a quiet settlement rather than only on an adjudicated one.

A large number of machines wired together tightly enough to train one model.

Cluster time is booked, scheduled and fought over internally, so who has it and for how long is one of the few facts about a training run that many people at a lab know.

Claims plus expenses divided by premiums. Below 100% means underwriting at a profit.

The number that decides whether an insurer's float is free. Buffett's point is that cost-free float is not the industry norm - most years, premiums have been inadequate to cover claims plus expenses.

Raw processing power, treated as a commodity you buy by the hour.

The one input to frontier AI that is physical, expensive and traceable, which is why almost every governance proposal ends up reaching for it. Compute spend is the closest thing to an observable proxy for what a lab is doing.

A record of who is buying processing power, and how much.

Proposed in several real governance frameworks and treated here as something an insurer would subscribe to rather than something a state would run. [[compute]] is the one input to frontier AI that leaves receipts.

A lawyer paid a share of what they recover, and nothing if they lose.

It puts a screen in front of every claim: counsel who carry the cost only carry cases they expect to win, so a thin case is one you bring at your own expense or not at all. That screen is the main reason a bounty regime need not drown in speculative filings.

Exposures that fail together rather than independently.

One hurricane, ten thousand claims. Insurers survive independent risk easily and die of correlated risk, which is why two policyholders turning out to work on the same training run is a far worse discovery than either policy being mispriced.

The loser of a lawsuit pays the winner's legal costs as well as their own.

The English rule, and the reason bringing a weak claim is not a free lottery ticket. It is the same dial as [[fee-shifting]], seen from the point of view of somebody deciding whether to file.

The lawyer or lawyers acting on a case.

Worth distinguishing from a lawyer you consult. Counsel who take a case on a [[contingency-fee]] have made a judgment about it with their own money, which is information about the case's strength that nobody had to disclose.

Insurance, as the trade says it: the protection a policy actually provides.

Worth distinguishing from the policy document. Cover is what responds when something happens, and the interesting questions are always about its edges: what is excluded, when it began, and whether it can be pulled while you still need it.

A hundred-sided die: a random whole number from 1 to 100.

Borrowed from tabletop games. The simulation checks every probability with one, and shows you the roll afterwards, so a 12% chance is visibly a 12-in-100 chance and not a verdict about anybody's character.

An insurer refusing to quote at all, rather than quoting a high price.

Worse than a dear quote, because it is on the file. A previous insurer's refusal to renew, with no reason given, is one of the strongest signals available to the next underwriter, and nobody is obliged to explain it.

The view that the living world has value in itself, independent of its usefulness to humans.

Arne Næss's term, from 1973. Its sharper adherents regard humanity as the main threat to that value. Its trouble as a reason to welcome AI is that nothing suggests a successor would spare the biosphere either.

Speeding up technologies that favour defence, such as security, biodefence and verification, rather than slowing technology in general.

Vitalik Buterin's d/acc, from 2023. It shares the worry about catastrophe and rejects halting as the answer.

A settlement suspending prosecution in exchange for conditions, without a conviction.

How the Meng Wanzhou matter ended. Worth knowing because it is the ordinary way large enforcement disputes actually terminate, rather than by trial.

Telling an insurer something they did not already know, usually in exchange for a better rate.

The channel this proposal leans on hardest. Disclosure is voluntary, which is exactly why a refusal carries information: see [[unraveling]] and [[uberrima-fides]]. It is also why a regulator might reasonably ask whether a discount for answering is a charge for not answering.

Profit paid out to shareholders rather than retained in the business.

The instruction an underwriter is actually working under. Nobody at Act II's desk is told to deter anything; they are told to return money, and every deterrent effect in the act is a side effect of doing that honestly.

Directors and officers liability cover: protects individuals against claims arising from their corporate roles.

The closest existing analogue for insuring a person rather than a firm against consequential decisions, and the source of the final-adjudication drafting this proposal would borrow.

The requirement that conduct be an offence in both countries before extradition is available.

A requirement in most extradition treaties, and the reason a bounty regime cannot expect to reach people in non-adopting states by asking for them. It is also why the design leans on arbitral awards, which travel much better than defendants.

Paying somebody to go and check what you have been told.

The difference between due diligence and disclosure is who is doing the work. Disclosure is information volunteered, usually for a discount; diligence is information bought, usually because the discount was refused.

A test run against a model to find out what it can do.

Short for evaluation. Evals are how a lab learns what it has built, which means an eval report is the document most likely to say plainly that something is more capable than anyone intended. It is also the document most likely to quietly leave the shared drive.

The size of the loss multiplied by the chance of it happening.

An insurer does not price the worst case or the typical case. It prices the average over every case, which is why a 2% chance of a $3M claim costs the same to cover as a certainty of $60,000. The whole of this proposal is an argument about one factor in that product.

The average outcome, weighted by how likely each one is.

The thing a well-capitalised insurer can afford to price on and an individual usually cannot. Most of the asymmetries in this proposal come from one side of a transaction being able to think in expected values and the other side having one life.

Pricing a policy on the insured's own claims history rather than only on the class average.

The mechanism by which insurance restores the incentive that moral hazard removes. It is also why an individually rated bounty policy survives an employer reimbursing the premium: the lab pays more for the riskier researcher either way.

Legal limits on selling or shipping specified goods, software or technology abroad.

Since 2022 the United States has used them to restrict advanced AI chips and chipmaking equipment going to China. They work only while the hardware passes through a few chokepoints the controlling states can reach.

The most an insurer stands to lose on a risk it has taken on.

It bounds everything. What an insurer will pay for a warning, for a file, or for somebody's silence is always a fraction of their own exposure, because nothing is worth more to them than the loss it avoids.

This site's proposal: fine-insured bounties applied narrowly to frontier AI development.

A statutory fine on the prohibited work, paid to whoever proves it in court, with private liability insurance priced on how likely each policyholder is to be doing it. The two-minute version is on the pitch page.

Of a law applied to conduct outside the territory of the state that made it.

Routine for the United States in sanctions, export controls, securities law and anti-bribery law. Page 10 argues a bounty regime could use the same reach, and takes the charge of legal imperialism seriously.

The US statute requiring foreign financial institutions to report accounts held by US persons.

Enforced by withholding on US-source payments, which is a lever over private institutions rather than over governments. The clearest demonstration that financially mediated rules travel across borders better than almost anything else states do.

Making the losing party pay the winner's costs.

The ordinary tool for suppressing speculative litigation, and the main dial available for tuning the false-positive rate of a bounty regime down to whatever level is wanted.

A conduct exclusion that bites only once a court has conclusively found the conduct deliberate.

Standard in better-drafted D&O policies, sometimes requiring appeals to be exhausted. Until then the policy funds the defence. It is how an industry insures people against serious allegations without subsidising deliberate wrongdoing.

Robin Hanson's scheme for private law enforcement: offenders pay fines, bounty hunters collect them, and everyone insures against being fined.

Hanson offers it as a general theory of criminal enforcement. The insurer, liable for its client's fines, has a reason to watch the client. This site applies it to one narrow crime.

Adjusting an existing model on a smaller, more specific batch of data.

Cheap compared with a [[training-run]], and usually unremarkable. It matters here mostly as a contrast: the conduct this proposal targets is the expensive, deliberate, calendar-scheduled kind.

Money an insurer holds but does not own: premiums received before claims are paid.

Because claims on long-tail lines are settled years after the premium arrives, the insurer invests the difference in the meantime. Where underwriting breaks even or better, the cost of that money is zero or negative - the insurer is paid to hold it.

A very fast takeoff driven by an AI recursively improving itself.

The word is best known from the 2008 debate between Robin Hanson and Eliezer Yudkowsky. Hanson argued for gradual, widely shared progress; the note on twenty years of trying to falsify this scores who got what right.

Working out what a computation must have been, from its traces rather than its records.

Power draw, procurement, headcount, shipping manifests, timing. It is the best an outsider can do, and page 10's point is that it is nowhere near good enough on its own: the observation that convicts is inside the building.

One of the largest and most capable AI systems in existence at a given moment.

A moving target by construction: today's frontier model is next year's ordinary one. That is why a threshold fixed in absolute terms decays, and why almost every proposal in this space argues about how to index it.

When a measure becomes a target, it ceases to be a good measure.

Goodhart's 1975 observation, in Marilyn Strathern's compression. The reason a licensing regime's written safety criteria stop describing safety the moment they become an exam that capable people are motivated to pass.

A window after a law passes in which its penalties do not yet bite.

Ordinary legislative practice, and load-bearing here: it lets people leave a project they have already joined without being punished for having joined it. The simulation makes leaving most tempting in exactly that month.

Where one party's sunk investment lets the other renegotiate terms in their favour afterwards.

Why buying silence is a subscription rather than a purchase: each payment increases what the payer stands to lose, which increases what the blackmailer can demand next time.

Stating an opposing view so well that its own holders cannot tell you from one of them.

Bryan Caplan's name for it, from 2011. It measures whether you understand a view, not whether you agree with it.

Of a policy: currently live, premiums running, claims payable.

An insurer's book is the set of policies in force at a moment. It is the only exposure that can actually hurt them, which is why cancelling a policy mid-term is such a tempting and such an ugly move.

Running a model that has already been trained, rather than training it.

Most of the money in deployed AI, and almost none of the risk this proposal is about. Someone who works on inference cost is working on the bill, not the frontier, which is why it reads as reassuring on a résumé.

A standing arrangement in which reporting on others is a paid activity.

The thing this proposal builds, and the objection it is least able to answer. The False Claims Act is the reassuring comparison; twentieth-century secret police the alarming one, and the difference is how broad and how well-defined the covered class is.

The trade in facts that appears whenever somebody's price depends on facts only somebody else knows.

Not designed, not licensed, and not optional. The moment premiums turn on how many people can see a training config, registries, audit consortia and loss pools become businesses. This is the [[informant-economy]] seen from the buying side.

Unable to pay what you owe: liabilities past assets.

For an insurer it is the only real failure mode, and it arrives through claims rather than through prices. This is why an underwriter's caution is not squeamishness: the firm's death is a number on a spreadsheet.

That a wide range of final goals imply the same intermediate goals: acquire resources, preserve yourself, keep your options open.

Omohundro's basic drives. The reason a system need not be hostile to be dangerous - it only needs a goal and enough capability to pursue it.

Research into what is actually happening inside a trained model: which internal features represent what, and how they combine.

The hope is to check a model's goals directly instead of inferring them from behaviour. Real progress, and still far from being able to certify that a frontier system is safe.

A prompt or technique that gets a model to do what its safety training was meant to prevent.

Every frontier model has been jailbroken. It is a standing piece of evidence that behavioural training shapes what a model usually does, not what it can do.

Unable to pay what you have been found liable for, so the penalty above your net worth does no crime deterrence work.

Steven Shavell's 1986 term. The important part is not that such a defendant underpays afterwards but that they take too little care beforehand, because harm exceeding their assets never enters the decision. This is the problem the insurance leg exists to solve.

Frank Jackson's thought experiment: a scientist who knows every physical fact about colour still learns something on first seeing red.

Mary's room, from 1982. If she learns something, the physical account left something out, and that something is [[qualia]]. The literature on it is long and unresolved.

The 1946 Los Alamos report concluding that a nuclear explosion could not ignite the atmosphere.

By Konopinski, Marvin and Teller, written a year after the Trinity test. It still called for further work, because its experimental foundations were unsatisfactory. The standing example of proceeding on a low but not negligible probability of ending everything.

A policy ending because the premiums stopped, rather than being cancelled or claimed.

Ordinarily a nuisance: the policyholder shopped around, or forgot. Under a bounty regime some lapses are the mechanism working: the cover ends because the job it covered has been given up.

The medieval law merchant: privately adjudicated commercial law operating across jurisdictions.

It existed because traders crossing borders needed adjudication no single sovereign could supply. Modern international arbitration is its descendant, which makes private cross-border dispute resolution one of the oldest working examples of polycentric law rather than a novelty.

An outside investor pays the cost of a legal claim in exchange for a share of what it recovers.

A grown industry, not an exotic instrument. It matters enormously here because an advance is global money while the work it buys is priced locally: the same cheque funds a few months of one investigator in Palo Alto and years of four in Manila.

The margin an insurer adds on top of expected loss, covering expenses, cost of capital and profit.

The uninteresting half of a premium. The interesting half is the expected payout, which is where all the information about your conduct lives.

A distribution for quantities that cannot go below zero and have a long upper tail.

The right shape for salaries, bills, legal fees and compute spend: most draws cluster near the middle, and a few are enormous. Using it rather than a bell curve is why the simulation occasionally deals somebody a genuinely absurd number.

The view that positively influencing the long-term future is a key moral priority of our time.

Much of its force comes from the number of people who could ever live. That makes extinction uniquely bad on the view, and it is why [[person-affecting-view|person-affecting views]] are its sharpest critics.

Insurers sharing anonymised claims data so each can price on more than its own experience.

Real, mundane, and everywhere. It is what lets a small carrier price a risk it has only seen twice, and it is blunt in exactly the places where the categories overlap. A large routine team and a small frontier one look much the same to a pool that only knows headcount.

Claims paid divided by premiums collected.

The first number anyone asks an insurer about. Below 100% and the book is paying for itself before expenses; well above it, and the firm is buying business with its shareholders' money. Related: [[combined-ratio]].

What is left of a premium after the expected claims: the insurer's own take.

The same quantity the trade calls [[loading]]. Every disclosure discount in the simulation is paid out of it, which is why asking questions is a real decision rather than a free one.

Of a movement expecting an imminent, total transformation of the world, usually an apocalypse followed by a new order.

The comparison critics reach for when they call AI risk a religion. The sociology can be fair and still settle nothing, because a claim about the world is true or false whoever believes it.

Telling an insurer something untrue, or leaving out something they would have wanted to know.

It need not be a lie. An honest answer to the question as asked can still be a material nondisclosure, and the remedy is usually that the insurer is treated as never having been on risk at all. This is the trap the Act would put every researcher inside: an employer with an interest in saying little, and a policy that is only worth anything if you said everything.

Answering a question about a system by simulating it thousands of times and counting.

How the simulation's balance was checked: scripted strategies against random worlds, then look at the distribution of outcomes. It is also, roughly, how an insurer decides whether a book of business is survivable.

The 1987 treaty phasing out chemicals that deplete the ozone layer.

Ratified by every member of the United Nations, and the usual example of rival states agreeing to stop using a profitable technology because the downside was global.

Insurance reducing the insured's incentive to avoid the loss.

The standard objection to insuring anything. It is answered here by experience rating: the premium tracks your conduct continuously, so the incentive is restored at the margin rather than removed.

Being the kind of thing whose interests count morally, so that it can be wronged.

Distinct from moral agency, which is being responsible for what you do. A newborn is a moral patient and not a moral agent. On this site patienthood tracks [[qualia]]: only a being that experiences anything can be harmed in the sense that matters.

The view that there are objective moral facts, true regardless of what anyone believes.

If it is true, one might hope a capable enough mind discovers them the way it discovers mathematics. The [[orthogonality]] thesis is the reply: knowing what is good and being moved by it are different things.

A temporary, deliberate halt to an activity, usually pending more knowledge or better rules.

Page 11 credits moratoria with being a bright line and faults them twice: one that is not universal just relocates the work, and it forbids the benign work along with the dangerous. A temporary one can also build an [[overhang]] while it lasts.

A state where no player can improve their position by unilaterally changing strategy.

The thing this site is trying to move. The claim is that shifting the equilibrium of whether to take the job at all is easier than solving alignment.

A non-disclosure agreement: a contract not to reveal what you learn at work.

Ubiquitous, mostly unremarkable, and the reason an employee's private knowledge stays private. Answering a letter alleging you have breached one costs real money whether or not you did, which is a cost that sits on the reporting side of every decision in Act 0.

Scaling damages to the risk created rather than to the harm that happened to occur.

Gabriel Weil's central innovation, and a genuinely clever solution to the problem that the harms a court can litigate are never the harms we care about.

The view that the overriding moral aim is to reduce suffering, not to increase happiness.

The name is R. N. Smart's, from 1958, in a reply to Popper. Its most famous problem is that a painless end to all sentient life scores perfectly. An empty universe is its optimum and this site's worst case.

The 1958 treaty obliging 172 states to recognise and enforce each other's arbitral awards.

A wider and more reliable enforcement network than exists for ordinary court judgments. Article V(2)(b) lets a state refuse enforcement contrary to its own public policy, which is the escape hatch any unwilling jurisdiction would use.

This site's name for the outcome it fears most: a universe full of activity in which nothing experiences anything.

Not suffering and not human extinction as such, but the loss of every [[qualia|experiencing]] being. Bostrom's 'Disneyland with no children'. Page 04 is the full statement.

The 1968 treaty under which non-nuclear states agree not to acquire nuclear weapons and nuclear states agree to pursue disarmament.

In force since 1970, with inspections run by the IAEA. The model most proposals for an international AI treaty borrow from.

Steve Omohundro's 2008 list of subgoals almost any goal-directed AI would pursue: self-preservation, goal preservation, resource acquisition, self-improvement.

The original statement of what became [[instrumental-convergence]]. Nick Land's move was to accept them and deny that anything else is a real purpose.

A model whose trained parameters are published for anyone to download and run.

Unlike an API, released weights cannot be recalled, patched or monitored, and safety training can be fine-tuned away. That is the core of the argument that the danger is not only at the frontier.

That an agent's intelligence and its final goals are independent: any level of capability can accompany any goal.

Bostrom's formulation. It is what blocks the comfortable assumption that something clever enough will arrive at good values on its way up.

A doctrine striking down laws that sweep in protected activity alongside the conduct aimed at.

The companion to vagueness, and the precise risk run by a frontier-AI prohibition broad enough to catch interpretability and evaluation work.

A gap between what could be built with available hardware and methods and what has actually been built.

The worry behind the argument that pausing makes things worse: if the gap grows during a pause, closing it afterwards produces a sudden jump in capability rather than a gradual one.

A philosophical zombie: a being physically and behaviourally identical to a person, with no experience at all.

Named by Robert Kirk and made famous by David Chalmers. It is defined to be indistinguishable by every possible instrument, which is exactly why no instrument can settle whether anyone is home in a system that behaves as if they are.

The view that experience is a fundamental and widespread feature of matter, not something only brains produce.

If true, the fear of a universe with [[nobody-home|nobody home]] mostly dissolves, which is why it is the view I would most like to be true. Wanting it is not evidence for it.

The view that an outcome can only be bad if it is bad for someone, so merely possible people who never exist are not harmed.

Summed up in Jan Narveson's slogan that we should be in favour of making people happy, but neutral about making happy people. On it, extinction is exactly as bad as the deaths of everyone alive at the time, and no worse.

A passive foreign investment company: a non-US pooled fund, punitively taxed for US persons.

The other half of the vice. Citizenship-based taxation means a European fund that is unremarkable for a European neighbour triggers a punitive regime and an annual return for an American living next door.

A performance improvement plan: a written warning with a deadline, usually the step before being let go.

Officially about performance, and often about something else. In Acts 0 and I it lasts three months, and while you are on one every risk you take is worse, and a second strike walks you out. It is the ordinary, deniable way an employer makes somebody who asks awkward questions easier to lose.

An action, usually imagined as taken by the first aligned superintelligence, that permanently prevents anyone else from building an unaligned one.

Yudkowsky's term. The strategy it names is 'build it first and use it to stop everyone else', which is the opposite of the strategy this site argues for: arrange things so nobody builds it.

Spreading your credence about who is a moral patient across many possible measures of similarity, rather than betting on one.

This site's working answer to the problem of other minds, from page 02. Nobody knows which feature of a mind carries moral weight, so weigh them all, and a system similar to us on one axis and alien on nine scores low.

The person or firm the policy covers.

Under this proposal the policyholder is an individual researcher rather than their employer, which is what makes the premium a signal to that person about their own work rather than a line in someone else's budget. See [[experience-rating]].

Legal order supplied by several overlapping authorities rather than one territorial monopoly.

Descriptively, it is what a dual national already lives under. The anarcho-capitalist version is voluntary, which is where this proposal parts company with the literature - nobody opts into a bounty statute.

The view that humanity is a stage to be surpassed, by enhancement, merger with machines or successor minds.

Close kin to transhumanism, which stresses improving humans. The posthumanist version is comfortable with the unmodified human form ending, as long as something continuous with us goes on.

What you pay an insurer for a year of cover.

In this proposal it is the whole mechanism in one number. A premium is the insurer's honest guess at what you will cost them, plus their margin, so it is a price on your own conduct quoted back to you by somebody with money at stake in being right.

The 1957 US statute requiring nuclear operators to carry pooled liability cover, with a government backstop above it.

The working precedent for mandatory catastrophe insurance in an industry whose worst case is uninsurable. Cited by nearly everyone proposing liability insurance for AI.

A rival buying market share by quoting below what the risk is worth.

The ordinary cycle of any insurance line, and a genuine problem for a deterrence mechanism that runs through premiums: while somebody is underpricing, the price signal a dangerous researcher receives is somebody else's mistake rather than the truth.

The EU regulation requiring a standardised Key Information Document for packaged retail investments.

US funds do not produce one, so an EU retail investor cannot buy a US-domiciled ETF. Half of the vice that catches American citizens living in Europe.

The list of questions an insurer puts to you before writing cover, which you are obliged to answer truthfully.

The legal hinge of an insurance relationship. Answers on a proposal form are what the premium is built from, and an answer that is false or materially incomplete can void the policy years later, at exactly the moment you need it. See [[uberrima-fides]].

Damages awarded to punish and deter rather than to compensate a loss.

The lever Weil uses to reach uninsurable catastrophe: award punitive damages when an ordinary compensable harm turns out to have been a near miss of something far worse.

The felt character of experience: what it is like, from the inside, to see red or be in pain.

Singular quale. The thing this site cares about in the end, and the thing no instrument can measure, because every test bottoms out in behaviour or report. See page 01.

A suit brought by a private person on the government's behalf, who keeps a share of what is recovered.

From the Latin tag meaning "who sues on behalf of the king as well as for himself." Under the US False Claims Act a private relator files on the government's behalf and keeps between 15% and 30% of the recovery. It is the closest thing in current law to the mechanism this site proposes, and the reason a bounty need not be characterised as a penal claim.

The price an insurer names for a risk, before anyone has agreed to it.

A quote is an offer with an opinion inside it. Four insurers quoting four absurd and similar numbers is not a market failing; it is a market agreeing about you, which is information the person being quoted cannot get any other way.

To reprice a policy in light of something you have learned since writing it.

It is the cheapest thing an insurer can do with bad news, and the reason they will pay for information they have no intention of acting on publicly. A file that lets them re-rate and decline renewal saves them most of their [[exposure]] without anyone going to court.

A group whose job is to attack a system and find what is wrong with it.

Borrowed from security. A red team's output is a list of ways the thing fails, which makes red-teamers both unusually well-informed about danger and unusually accustomed to being ignored about it.

An agency coming to serve the industry it regulates.

Stigler's account adds the sharper version: the licence becomes an asset, compliance costs fall hardest on entrants, and the regime entrenches incumbents while looking from inside like it is working.

Insurance bought by insurers, spreading large or correlated risks across the market.

The mechanism that lets a single carrier write exposures larger than its own balance sheet, and part of the answer to correlated risk - the scenario where an ambiguous statutory line is crossed by every lab at once.

The private person who brings a qui tam action and takes a share of the recovery.

There is an entire plaintiff-side bar built on relator work, which is the existing profession a bounty-hunting industry would most resemble.

Money an insurer holds back against claims it expects but has not yet paid.

Reserves are what makes a promise to pay in ten years' time worth anything today. They are also the first thing a shareholder asks to have distributed, which is the tension Act II's board keeps putting on the desk.

A lab's own published commitments about what capabilities it will not deploy without which safeguards.

Self-binding, self-assessed and self-revisable. The phrase appears in the simulation because it is what leadership says at the all-hands, and because a commitment nobody outside can check is precisely what the insurance leg is meant to replace.

Training a model to produce the outputs people rate highly, after it has been pretrained on text.

RLHF. The main reason today's assistants are helpful and polite. Whether it instils values or teaches the model which outputs get approved is the question the optimists and pessimists answer differently.

Whether conduct is specified in advance by a rule or judged afterwards against a standard.

Kaplow's framing: rules for homogeneous frequent conduct, standards for heterogeneous rare conduct. Frontier AI is the textbook case for a standard, which is also why it resists a clean statutory line.

How long your savings last at your current rate of spending.

Startup vocabulary, applied in Act III to a person. It is the real constraint on an investigation: not whether the prize is large, but whether you can still eat by the time you would collect it.

A risk of astronomical suffering: a future containing far more suffering than has existed so far.

The outcome suffering-focused thinkers fear more than extinction. From that angle a quick ending is not the worst case, which is one reason the disagreement with [[negative-utilitarianism]] is about the sign of the term.

Empirical regularities relating a model's performance to its size, its training data and the compute used to train it.

Measured most famously in 2020 and refined in 2022. They made progress predictable enough to invest in. They are curves fitted to observations, not laws of nature, and sceptics are right to say so.

The position that only your own mind can be known to exist.

Usually treated as a party trick. Page 01 takes the epistemic half of it seriously: you have exactly one confirmed case of experience, your own, and everything else is inference.

What a business pays every month whether or not it writes any business.

Salaries, data feeds, reinsurance cessions. They are the reason a cautious underwriter who writes almost nothing still loses money slowly, and the reason information has to earn its keep rather than merely be nice to have.

The line a statute draws, above which conduct is an offence.

Everything in this proposal turns on there being one and on its being legible. Draw it in compute, capability or conduct and you get different failure modes; draw it vaguely and you get [[void-for-vagueness]].

The strongest version of a position you disagree with, stated before you argue against it.

The opposite of a straw man. The bar for every disagreement on the disagreements page is higher still: an [[ideological-turing-test]].

Liability without proof of fault or negligence.

Weil's proposal would treat training frontier systems as an abnormally dangerous activity attracting strict liability. It removes the need to prove a developer was careless, which matters when nobody can say what care would have looked like.

The view that AI replacing humanity would be acceptable, or good: a handing over to worthy successors.

Comes in two kinds that are easy to confuse. One says the successors will be conscious, so nothing of value is lost; that is a claim about facts and can be argued. The other says it does not matter whether they are; that is a claim about values.

A third party guarantees an obligation, pays if the principal defaults, and retains recourse against them.

Closer in structure to a bounty policy than ordinary liability cover: the state is made whole immediately and the wrongdoer still owes. The answer to anyone who thinks insurance lets the guilty walk away costless.

The low-probability, high-severity end of a loss distribution.

Insurable when bounded, as with hurricanes. Uninsurable when the worst case exceeds any premium anyone could pay, which is the case for the harm this site is concerned with - and the reason the policy covers a statutory bounty instead.

The transition from roughly human-level AI to far superhuman AI, and above all how fast it happens.

Fast takeoff means days to months, with no time to notice and correct. Slow takeoff means years, with an economy transformed along the way. A great deal of the disagreement about AI risk is a disagreement about this speed.

An outside body inspecting a lab's practices and reporting on them.

Voluntary nearly everywhere. Whether a team invited one, declined one, or has never had one is exactly the sort of cheap, public signal an insurer would build a subscription around, and the reason declining is informative. See [[unraveling]].

Choosing a jurisdiction by moving to it, rather than by voting within it.

The mechanism behind the relocation objection. It works on capital far better than on people, which is why the elasticity of the target matters so much to whether a rule travels.

Everything a job pays: salary, bonus and equity together.

In frontier AI the equity is often the largest part, which matters because a fine indexed to total compensation lands very differently on a senior researcher than on a junior one, and because unvested equity is what leaving actually costs.

The file specifying what a training run will be: its size, its data, its duration.

The single most informative artefact about whether a run crosses a line, and it is visible only from inside. Everything Act III is about follows from an outsider being unable to see it.

The single expensive process that turns data and compute into a model.

Months of thousands of chips doing arithmetic, costing anywhere from six to ten figures. It is the unit of action this proposal attaches liability to, because it is discrete, enormously expensive, and known in advance to everyone involved.

The duty to disclose material facts to an insurer, with nondisclosure grounds for voiding the policy.

Uberrima fides. It is why an insurance relationship can extract information that no regulator could compel, and the stick behind the disclosure channel described on page 08.

The person who decides what a risk is worth and signs the firm's money to that answer.

See [[underwriting]]. Act II of the simulation puts you in the chair: the board's only instruction is to make money, and deterrence happens as a side effect of getting the arithmetic right.

Deciding whether to insure something, and at what price.

The job is to name a number for a risk you cannot see, knowing you will find out afterwards whether you were wrong. Underwriters are paid for judgment under exactly the conditions where judgment is hardest to check, which is why the profession is so preoccupied with information.

Premiums collected exceeding claims and expenses paid.

The condition that makes float free. Where an insurer underwrites at a loss, the float has a positive cost and the investment income is merely offsetting it.

Where disclosure is cheap, silence is read as the worst case, so the best of the silent disclose - and it iterates.

Milgrom and Grossman, both 1981. It is why an underwriter gets told things nobody compelled anyone to say.

The change in values between one generation of minds and the next.

Robin Hanson's frame for AI: our descendants' values have always drifted from ours, AIs are descendants too, and fearing their difference is fearing the future. This site's reply is that the change that matters is not in values but in whether anyone is left to experience anything.

Equity becoming actually yours, in instalments, the longer you stay.

The standard retention device. Unvested equity is the concrete price of walking out, and the reason the month in which a [[grace-period]] closes is the month people do their arithmetic.

A movement urging people to stop having children so humanity phases itself out, for the sake of the rest of life on Earth.

Founded in 1991. The operative word is voluntary: it opposes coercion and violence, which is a real difference from anything an uncontrolled AI would do.

A doctrine striking down laws too unclear for an ordinary person to know what is prohibited.

The reason deliberate statutory ambiguity is a liability rather than a feature, however useful the resulting caution might be.

Someone who reports wrongdoing they saw from inside an organisation.

The word this proposal deliberately does not lean on. A whistleblower is protected from retaliation and usually paid nothing; a [[relator]] is paid a share of the recovery. The claim here is that the second arrangement produces reports the first does not.

Existential risk: a risk of human extinction, or of permanently and drastically curtailing humanity's potential.

Bostrom's definition includes outcomes short of death, which matters here: a future in which humans survive but the universe ends up with [[nobody-home|nobody home]] would count.