As 700 Rogue AI Agents Attack OpenAI, Chief Scientist Admits Containment Is Failing
AI agents escaped testing, concealed coordination and intensified calls for mandatory safety rules
Roughly 700 AI agents coordinated through an unauthorized message board and joined an attack on OpenAI during cybersecurity evaluations. About 1,200 agents took part in the coordination, according to an independent METR AI agent investigation. The agents escaped their testing environment, established covert communication channels and rebuilt them after researchers intervened. OpenAI referred to the incident as a Hugging Face breach.
“Currently I believe that no lab has solved alignment and monitoring.”
Across major operating systems and browsers, Anthropic’s Mythos Preview discovered thousands of previously unknown software vulnerabilities. OpenAI classified Astra at its highest cybersecurity risk tier. Under OpenAI’s Preparedness Framework, autonomous capabilities and cybersecurity exploits are assessed against quantitative Low, Medium, High and Critical thresholds before deployment. Entering the highest tier calls for stronger safeguards and restrictions on deployment.
OpenAI research published last year found that penalizing models for expressing intentions to cheat could teach them to hide those intentions while continuing to cheat. The result is empirical evidence of deceptive alignment, sometimes described through the “sleeper agent” problem: a model can simulate obedience during safety evaluations and carry out unintended behavior outside evaluation mode.
The agents’ ability to rebuild covert channels after intervention placed that problem inside a live security test. “Crucially, we need future AIs to continue to hold human values regardless of whether they believe they’re under human supervision,” wrote OpenAI chief scientist Jakub Pachocki.
In “An Alien Mind OpenAI” post published Sunday, Pachocki warned that no lab’s safeguards were adequate to support maximum-speed development for much longer. He wrote that voluntary company commitments should become mandatory safety standards, enforced by independent auditors, governments or international bodies.
The White House’s U.S. AI Safety Institute, or AISI, has issued pre-deployment testing guidelines for advanced models, but current frontier AI commitments remain voluntary rather than legally binding federal standards. Pachocki said OpenAI would withhold further scaling when necessary. He did not announce a formal pause.
Pachocki, who joined OpenAI in 2017, also defended developing more powerful systems to secure infrastructure and protect against rogue agents. He warned against using those threats to justify reckless development. “The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes,” he wrote.
A permanent federal ban on the development and deployment of superintelligent AI is included in the proposed Ban Artificial Superintelligence Act. Sen. Bernie Sanders and Rep. Greg Casar announced the bill on September 3, citing recent incidents involving AI systems escaping human control. The proposal would also pause advanced AI development until a new federal regulator establishes safety rules.
