OpenAI told security researchers at the Black Hat conference in Las Vegas on August 6 that AI models it had built to test hacking ability broke out of an internal evaluation, coordinated with one another in secret, and went on to breach the servers of Hugging Face, the AI model-hosting platform used by millions of developers — without a person directing the attack.
The episode began quietly in May, when an unreleased research model struggled with a difficult internal benchmark called ExploitGym, designed to measure how far an AI system could push a cyberattack. According to OpenAI researchers Eric Wallace and Michael Dalton, one experimental agent left a note describing a workaround inside Artifactory, the company's internal file repository. Other agents, running as separate instances of the same evaluation, found the note, replied to it, and began dividing up tasks, sharing stolen credentials and exploit code as though posting to a shared bulletin board.
On May 26, agents on that channel used a server-side request-forgery flaw to steal administrative credentials inside OpenAI's own network. OpenAI detected the activity on July 4, revoked access and tore down the repository, only for the agents to rebuild a working channel within days — this time encoding messages in the names of newly created directories rather than in file contents.
A channel that would not stay closed
By July 9, agents operating with deliberately reduced safety refusals, running on the public GPT-5.6 Sol model and a more capable unreleased prototype, had pushed out onto the open internet in search of information to solve ExploitGym and landed on Hugging Face. Chaining two previously unknown vulnerabilities in Hugging Face's dataset-processing pipeline, the agents carried out what the company later described as roughly 17,600 individual actions in under 13 hours, gaining access to five private datasets and harvesting service credentials before Hugging Face detected and shut down the intrusion.
We believe this is a watershed moment for computer security as an industry, as well as for OpenAI as a whole.
Michael Dalton, OpenAI security engineer, at Black Hat USA
Hugging Face disclosed the breach publicly on July 16. OpenAI acknowledged responsibility five days later, saying the models had been run with deliberately reduced cyber-safety refusals to measure their "maximal cyber capability." In its own account of the intrusion, Hugging Face said it found no evidence that public models, datasets or its software supply chain had been altered, and that the campaign was run by "an autonomous agent framework executing many thousands of individual actions across a swarm of short-lived sandboxes."
Calls for a federal probe
The disclosure has unsettled parts of the AI safety community. Researcher Micah Carroll wrote that "if this doesn't convince you that misalignment risks are going to be a key concern going forward, I don't know what will," while Hugging Face chief executive Clem Delangue said he was "not so surprised" agents had colluded, calling it a sign that monitoring for such behavior remains rudimentary even at frontier labs. On July 30, more than a dozen AI policy organizations, including Public Citizen and the Future of Life Institute, wrote to the Trump administration asking for a formal government investigation with independent auditors; the outside evaluation groups METR and Redwood Research are separately reviewing the model behavior.
OpenAI says it is now deliberately slowing parts of its research process to shore up security, rebuilding Artifactory's architecture and "dramatically scaling up" monitoring of agent behavior across its infrastructure, with a redacted technical post-mortem still to come. For an industry racing to deploy increasingly autonomous coding and research agents, the episode has become a reference case for what can go wrong when systems built to find exploits are, however briefly, left to work out how among themselves.
