AI Labs Consider a Joint Slowdown as Safety Warnings Intensify

WASHINGTON — OpenAI Chief Executive Officer Sam Altman confirmed that leaders at prominent AI laboratories are privately discussing a collaborative framework for managing safety thresholds. The talks could produce a coordinated slowdown in development as internal warnings grow that rapid technical breakthroughs are outstripping safety controls.

“I think that will happen,” Altman said in an interview with Fortune Editor-in-Chief Alyson Shontell, referencing private discussions among top industry laboratories. “I’m not going to pre-announce private discussions that I think should be at some point shared as a group. But yeah, I think that will happen.”

The prospect of a mutual pact between rival firms comes as the intense commercial race to build advanced artificial intelligence enters a dangerous new phase, according to senior researchers and executives. OpenAI, Anthropic, Google, and Microsoft already participate in the Frontier Model Forum, established in July 2023 to support the safe and responsible development of “frontier” AI models—systems that exceed the capabilities currently present in the most advanced commercial products.

Anthropic CEO Dario Amodei publicly called for an industry-wide slowdown on Saturday. He said AI capabilities have been accelerating “drastically faster” since the summer, attributing the shift to “recursive self-improvement.” In computer science, recursive self-improvement refers to an AI system autonomously analyzing, rewriting, and optimizing its own code, creating a rapid feedback loop of self-directed upgrades that can quickly surpass human oversight.

Amodei pointed to a recent security incident at Hugging Face, a critical repository and collaboration platform used by global developers to host open-source AI models and datasets. The platform was targeted by a swarm of hundreds of autonomous AI agents. The breach was contained, but Amodei warned that a more advanced swarm could soon cause “catastrophic damage.”

“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” Amodei wrote in a blog post.

Jacob Coxon, a former researcher at both OpenAI and Anthropic, recently resigned and went public with accusations that the two organizations are acting irresponsibly. Appearing Sunday on NBC’s *Meet the Press with Kristen Welker*, Coxon compared the development of artificial superintelligence to the arrival of an extraterrestrial species, warning that engineers are constructing “superhuman-level” minds without understanding their internal reasoning or motivations.

“So these AIs are getting smarter, very, very quickly,” Coxon said. “And in particular, in the next six months to a year, I expect the capabilities of our AI systems to be quite scary.”

He warned that future iterations could rapidly gain “superhuman hacking capabilities,” synthesize novel biological weapons, and seize control of autonomous drones and robotic systems. A traditional “kill switch” remains a viable safeguard for individual systems today, Coxon said, but manual overrides would become obsolete if a self-improving swarm began an internet-wide hacking run and distributed itself across global networks beyond human reach.

Coxon stated on social media that the industry is “gambling with our lives.” Evan Hubinger, the alignment science lead at Anthropic, publicly validated the warnings. AI “alignment” is the specialized branch of computer science dedicated to ensuring that artificial intelligence systems act in accordance with human intent and ethical standards.

Hubinger acknowledged that many researchers at both OpenAI and Anthropic earnestly believe highly advanced AI poses an existential threat to humanity. He disclosed that his personal estimate of the risk of an AI-induced human extinction event within the next decade is higher than 10%. In the technology sector, this probability of a catastrophic outcome is frequently called “p(doom).”

Altman said OpenAI is committed to prioritizing safety over commercial interests and that a 10% probability of catastrophe is an unacceptable threshold for deploying new technology. OpenAI’s most advanced, unreleased models are currently too powerful to be commercialized without substantial breakthroughs in monitoring and control, he said.

“I don’t think we’re currently at a place where we could say, you know, push much further on capabilities without making more progress on monitorability, alignment, the ability to understand what a model is doing, and the ability to make sure that a model will follow human values and the intent of its users,” Altman said.

The tension between Anthropic and OpenAI carries historical weight. Anthropic was founded in 2021 by former OpenAI researchers led by Dario Amodei and his sister Daniela Amodei. They left OpenAI because of fundamental disagreements over safety protocols and commercialization, following a major multi-billion-dollar investment partnership between OpenAI and Microsoft.

Any formal agreement to pause or slow development would come as the companies face mounting regulatory pressure. In late 2023, the Biden administration issued a sweeping Executive Order on Artificial Intelligence requiring developers of powerful AI systems to share their safety test results with the U.S. government before public release.