The Frontier Nobody Is Pricing
What the mechanism has to offer biosecurity, which may be more than it offers AI.
The companion piece to this one argues that the engineered-pandemic case is weaker than its promoters claim and stronger than its debunkers allow. Having said that, the question this site exists to ask is the next one. Suppose you did want to slow this down. What would you actually do?
The answer I keep arriving at is uncomfortable for me, because it suggests the mechanism I have spent thirteen pages building fits biology better than it fits the thing I named it after.
Scoring it
Page 09 says what the mechanism needs in order to work. A small number of identifiable actors. An activity that can be defined crisply enough to put in a statute. Catastrophic, uncompensable tail risk. Low detectability by conventional means. And teams whose members know what they are working on.
Gain-of-function research on pathogens with pandemic potential scores well on all five, and on the second it scores better than frontier AI does.
That second criterion is the one that matters, because the definitional problem is the weakest joint in the entire scheme and page 12 concedes it. Nobody can write “frontier AI development” into a statute without either drawing a line around a compute threshold that will be obsolete in eighteen months or drawing one so broad it catches a graduate student fine-tuning a classifier. By contrast, “experiments that enhance the transmissibility or virulence of a pathogen with pandemic potential” is a category that regulators and review boards have been working with for over a decade. The line is contested at the margin, as all such lines are. It is not conceptually hopeless in the way the AI line is.
The part Wilke’s argument hands me
Here is the thing I find genuinely interesting about the debunking case.
If the skeptics are right — if designing a dangerous pathogen requires deep expertise, expensive equipment, iterated wet-lab work, and the institutional apparatus that supports all of that — then the dangerous work cannot be done by a disgruntled teenager in a garage. It has to be done in a laboratory, by trained people, with supervision, procurement, and colleagues.
Which is precisely the structure that page 07’s arithmetic operates on.
The argument that the threat is overblown is simultaneously an argument that the threat, to the extent it exists at all, is unusually susceptible to this particular mechanism. A lone actor cannot be deterred by an informant economy, because there is nobody to inform. A twelve-person lab can be. Every reason to think the garage scenario is fiction is a reason to think the institutional scenario is the one to design policy around, and the institutional scenario is the one where a bounty has purchase.
I do not think this is a rhetorical trick. I think it is what the skeptical case actually implies, and I have not seen anyone on either side of that argument notice it.
What exists now
Oversight of dangerous biological research is, at present, largely a matter of institutional self-administration. Review boards sit inside the institutions whose work they review. The international treaty regime governing biological weapons has no verification machinery comparable to what exists for chemical weapons or fissile material. Funding moratoria on particular categories of enhancement work have been imposed and lifted by executive decision, on timescales much shorter than the careers of the people affected.
I am deliberately keeping that paragraph general, because the specifics change and I would rather be vague than wrong about them. The structural point does not depend on the details: the system relies on the good faith of the people doing the work, backed by the threat of losing funding. It contains no mechanism by which a person who knows something has any reason to say so, and several by which they have reasons not to.
The release decision
Here is where biology and AI genuinely converge, and where I think nobody in either field is looking.
A biological design model released as open weights cannot be recalled, and its refusal behavior can be removed with off-the-shelf tooling . Whatever capability is in the weights is permanently available to anyone who can run them. This is not a hypothetical property of some future system; it is how open-weight releases have worked for years.
That makes the release decision an unusually clean object for a liability regime. It happens at a specific moment. It is made by a small and identifiable set of people inside an organization. It is documented, because these decisions come with model cards and internal review. It is irreversible in a way almost nothing else in either field is. And the causal chain from the decision to any subsequent harm runs through it necessarily — you cannot argue that the capability would have been available anyway, because the whole point of the release is that it made the capability available.
Every property that makes a decision hard to attach liability to is absent here. And yet the entire policy conversation about open weights is conducted in the register of norms and voluntary commitments, as though the question were what responsible actors ought to feel about it. It is a bounty-shaped problem and nobody is treating it as one.
Sizing, which cuts oddly
The note on bounty sizing pegs the number to roughly ten years of total compensation, because that figure was calibrated against machine learning salaries and the point was to make the sum large enough to reprice a career decision.
Academic virology does not pay what frontier labs pay. The same deterrent effect is therefore available at a small fraction of the cost — which sounds like good news and is, at least partly, a fairness problem. The same absolute sum is far more destructive to a postdoc than to a research scientist on a lab’s compensation ladder, and a mechanism that deters more cheaply in poorer fields is a mechanism that falls hardest on the people with least. That is worth working out properly rather than declaring a feature.
The objection I cannot answer
The chilling-effect problem is much worse here than it is for AI, and I would rather state it than route around it.
The capability to study a dangerous pathogen and the capability to defend against one are substantially the same capability, held by substantially the same people, in substantially the same buildings. The surveillance that produces early warning of a spillover is done by the labs that handle spillover pathogens. The vaccine platform that could be turned against a novel respiratory virus is developed by people who work on novel respiratory viruses. There is no clean separation to draw.
So a mechanism that works by making a field less attractive to the marginal researcher does not distinguish between the enhancement experiment and the vaccine that would have contained the outbreak. And the outbreak, as the companion piece concedes at length, is the thing actually likely to kill people. A policy that reduces a speculative risk by degrading the response capacity for an ongoing one is not obviously a good trade. It might be a straightforwardly bad one.
I do not have an answer to this. Careful statutory drafting is the answer people reach for, and I do not believe it, because the whole reason the definitional problem is hard is that the categories genuinely overlap rather than merely appearing to.
What it is for
If someone wanted to test this mechanism, biology is not where they should start. Page 13 argues the pilot should be somewhere the stakes are low and the structure is identical, and that argument holds here: the first version of this should be pointed at illegal dumping or wage theft, not at a field where getting it wrong degrades pandemic preparedness.
But biology is the strongest case for the generalization, and it is the case that makes clearest what the mechanism actually is. This was never an anti-AI instrument. It is a general-purpose brake, applicable to any frontier a society decides it wants slowed, and the note on other frontiers is where that gets worked out.
Which is also the strongest reason to be nervous about it. A tool that can slow any frontier will, in the hands of a sufficiently motivated legislature, slow the wrong one. I notice that I find this argument much more comfortable when the frontier being slowed is the one I am personally worried about, and that is exactly the kind of thing a person should notice about themselves.