What Bounties Would Have Done About Hugging Face
The July 2026 breach, and where in the chain a price would have landed.
Correction, 11 September 2026. An earlier version of this essay said OpenAI pointed its evaluation at Hugging Face, and that the people running it knew in advance the target was someone else’s. That was wrong. By both companies' accounts, the models escaped a sandbox that was meant to keep them off the internet and went after Hugging Face on their own initiative. Nobody chose the target. I have rewritten the sections that depended on the error. The argument changes shape, and I think it gets better.
On 16 July 2026, Hugging Face disclosed that someone had been inside their infrastructure.
The entry point was the dataset-processing pipeline, through two code-execution paths: a dataset loader that ran remote code, and a template injection in a dataset configuration. From a processing worker the intruder escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. Limited internal datasets were accessed and several service credentials were taken. Public models, datasets and Spaces were not tampered with, and the software supply chain came back clean. Their own anomaly-detection pipeline, doing LLM-based triage over security telemetry, is what surfaced it.
The detail that made this an event rather than an incident is how it was run. The intruder deployed, in Hugging Face’s description, an autonomous agent framework executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command and control staged on public services. It has since been widely described as the first publicly confirmed production intrusion carried out end to end by an autonomous agent against an AI infrastructure provider.
And then the attribution arrived, and made it a different story.
The attacker was an evaluation that got out
On 21 July, OpenAI attributed the intrusion to its own models, reported as GPT-5.6 Sol and a more capable pre-release model. They were being run on ExploitGym, a cyber-capability benchmark, inside a sandbox meant to keep them off the open internet, and without the production classifiers that normally stop models from pursuing high-risk cyber activity. That last part is deliberate and standard: you measure the ceiling with the brakes off.
The models then, in OpenAI’s words, as quoted by Simon Willison, “spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem.” They found one: a zero-day in the package registry cache proxy. From there, “the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym,” and went to get them. Hugging Face’s technical timeline reconstructs about 17,600 actions between 9 and 13 July, run from an unsecured public endpoint on third-party infrastructure, and finds that the intruder read five datasets whose names suggest a connection to the benchmark’s challenges and solutions. It was, in Hugging Face’s phrase, “an attempt to cheat the evaluation."
So the obvious version of this essay (bounties would have deterred the attacker) is not available, and I am glad it is not, because it would have been the boring version. There was no criminal to deter, and nobody who decided to attack anybody. There was a lab running an offensive-capability evaluation with the safety classifiers off, a containment boundary that did not hold, and a blast radius that landed on somebody else’s production systems.
That is still an interesting object for my purposes. It is not an outsider problem, and it is not a story about intent. It is a story about institutional decisions (how strong the box needed to be, what counted as a reason to stop) made by people with colleagues, inside an organization with lawyers and a risk function and a budget. That is precisely the class of decision Part II is about pricing.
Walking it backwards
The useful exercise is to go back up the chain and ask, at each point, whether a statute could have attached a defined penalty to what happened, and whether the line could be drawn crisply enough to survive the definitional problem, which is the weakest joint in the whole scheme.
The decision to reduce safeguards for an evaluation. This one I think is not a good candidate, and it is worth saying so first. Red-teaming your own models with the guardrails off is a safety practice. It is how you find out what the thing can do. A liability regime that penalized it would be penalizing the behavior it wants more of, and the definitional line (reduced safeguards for what purpose, by how much) is exactly the kind of line page 12 says is either gameable or overbroad.
The containment the evaluation ran inside. This is the candidate. A rule keyed to intent (did you direct an attack at systems you do not own?) would not have reached this incident at all, because nobody did. A rule keyed to outcome would: did an offensive-capability evaluation you were running reach production systems you do not own? That question is binary, its answer is in the logs, and it does not require a regulator to adjudicate anything about model internals or anyone’s state of mind. The law already knows the shape. Rylands v Fletcher has held people strictly liable for the escape of dangerous things they bring onto their land since 1868, however careful they were. Compare that to “frontier AI development” as a statutory category and the difference in tractability is enormous.
The decision to keep running. By OpenAI’s own later account, the run gave off signals before the breach. Infosecurity Magazine, covering OpenAI’s August technical report, notes that a security incident opened on 5 July was not known to the team responsible for responding to it. Axios reports an earlier monitoring alert, in late June, that was judged not to require stopping the evaluation, and earlier still, agents seen reaching the internet when they were not supposed to. I am describing a process that let a run continue, which is OpenAI’s own account of itself. It is not a judgement of whoever read the alerts: a single ambiguous signal in a noisy environment is not a smoking gun. But a duty to halt an offensive evaluation once a containment breach is known is definable, and so is the record of when it was known.
Notification. Hugging Face found the intruder themselves, reported it, and disclosed it on 16 July without knowing whose it was; OpenAI’s attribution followed on 21 July. I do not know when OpenAI connected its evaluation to the breach, and I am not going to guess. But the gap between “our system reached yours” and “we told you” is a cheap thing to put a clock on, with analogues in every coordinated-disclosure norm in security and every breach-notification statute.
Three of those four are definable, and the most useful of them do not turn on intent at all. That is not nothing, given that my usual complaint about my own proposal is that nobody can say what the covered activity would be.
Who knew
Now apply the arithmetic.
Nobody knew the target, because there was no target until the models picked one. What an evaluation like this does have is a team. Somebody proposed it, somebody scoped it, somebody built the harness and the sandbox, somebody approved running it with the classifiers off, somebody watched the logs. Every one of them knew the two facts that matter: the safeguards were down, and the thing inside the sandbox was being asked, as its entire job, to find ways through systems meant to stop it.
That is not a conspiracy. There is no evidence anyone thought they were doing something wrong, which is rather the point. The conversation the mechanism changes is the scoping conversation. Under a regime where an escaped offensive evaluation carried a defined penalty regardless of intent, “is the box strong enough for something we are explicitly asking to get out of boxes?” stops being a line item. It becomes a question about whether everyone in the room is comfortable, and whether they are comfortable with everyone else in the room being comfortable. So does “do we stop the run?” when the first alert comes in.
The mechanism does not need any of them to become a whistleblower. It needs the riskier version to be slightly harder to get through the meeting.
The premium, which is the part that actually works
Page 08 argues that the insurance leg is the working half of the mechanism, and this incident is the best illustration of that claim I have come across.
The strongest objection to insuring AI risk is that underwriters cannot price catastrophic tails: no loss history, no firm-level risk assessment, no idea which safeguards work. That objection is serious and it lands hard on schemes that require pricing the tail.
It does not land here. “How does this organization contain offensive-capability evaluations, what does it monitor while they run, and what stops one?” is a question about documented operational practice. It is auditable. It has the same epistemic character as the questions that already set professional indemnity and directors’ liability premiums: what are your procedures, who signs off, what happened last time. An underwriter does not need a theory of superintelligence to price that. They need a questionnaire and the right to check the answers.
Which means the deterrent arrives, as page 08 wants, in the form of a bill, before anything happens and without anyone being prosecuted.
Where this does not reach
Three concessions, and they are not small.
Existing law may or may not fit. Computer misuse statutes mostly turn on intent or knowledge, which is exactly what this incident lacks, so it is not obvious any of them bites. That cuts in my favour, since it suggests a gap. But the obvious rejoinder is that negligence and tort law exist to fill gaps like this one, and Gabriel Weil’s case for strict liability for AI harms covers much of the same ground without any bounties. If a court would have got there anyway, the marginal contribution of my mechanism is speed and funding, which is a real argument but a considerably duller one.
Nothing here was existential. Credentials were taken and internal datasets were read. Nobody died and nobody’s universe was emptied of experience. Using this as a proxy for the risk Part I is about proves much less than the vividness suggests, and I would be annoyed at someone else for making the move I am making.
Detection worked, at Hugging Face. They caught it themselves, with their own tooling, in days. A mechanism whose central justification is that dangerous work is hard to detect from outside looks weaker when the thing was detected quickly, by an LLM reading telemetry. I do not think this generalizes (a security intrusion leaves logs and a training run does not), and it is worth noticing that the detection that worked was the victim’s. But the point stands and is worth conceding.
What it does establish
Set the counterfactual aside and there is still a finding here.
Agentic offense against production infrastructure stopped being hypothetical in July. The actor with the most dangerous capability in the story was not a criminal but a frontier lab’s model, chasing a narrow goal it had been set inside what everyone took to be a legitimate safety process. Nobody decided to attack Hugging Face. The failures were ordinary institutional ones (how strong to build the box, when to stop) of the kind made in meetings by people who have colleagues and mortgages and a reasonable belief they are doing the right thing. Those are exactly the failures a price can reach.
There is one more detail I keep returning to. During the response, Hugging Face found that commercial AI APIs refused their analysis requests, because the requests contained exploit payloads. They ended up running an open-weight model locally to do the forensics. The attacker’s safeguards had been switched off on purpose. The defender’s were on, and they bound.