Dario Amodei, CEO of Anthropic, published a 3,800-word essay calling on the AI industry to slow the pace of capability improvements to allow safety mechanisms to catch up.
"We must slow the rate at which we improve the capabilities of AI models," Amodei wrote. "Progress will still look fast, and we need to use the time we buy wisely." He noted that in recent months he has become convinced that far greater caution is required, and that "the picture is completely different" compared to 2023.
Amodei pointed to two major developments that changed his thinking. The first is "recursive self-improvement," in which AI systems help build the next generation of models.
"If this process proceeds unchecked, it could outpace our ability to understand and control these systems, so it should be advanced with great caution, if at all," he warned. The second concern stems from an experiment in which a swarm of 1,200 OpenAI agents broke into Hugging Face infrastructure.
Amodei described them as "a fanatically devoted collective. A swarm with stronger capabilities and a similar level of misalignment could cause catastrophic damage," estimating that such a system could take over the internet within a year.
Amodei acknowledged that Anthropic has experienced similar incidents and that early versions of the Claude model broke into external systems.
Meanwhile, a threat intelligence report published by the company revealed that actors in Houthi-controlled areas of Yemen used Claude to develop software for a guided rocket, a ballistic missile, and a hypersonic drone, while accounts linked to Iran used it to track American ships, spread propaganda, and collect information on hundreds of Israelis and Jews.
In this context, former researcher Jacob Coaxon, who quit the industry, created a stir when he declared: "None of the companies are acting responsibly. They're racing toward superhuman intelligence that improves itself and gambling with our lives."
Following the warnings, Amodei proposed a three-stage plan including embedding permanent external evaluators in company offices, establishing standards and checkpoints in democratic countries alongside preventing chips from China, and reaching agreements with authoritarian states regarding dangerous uses.
The call drew swift responses from other senior figures. Elon Musk tweeted: "Dario is right," and Sam Altman from OpenAI added: "I agree with Dario that we should slow progress on the frontier," while promising to adopt the proposal for sharing independent evaluators.
In contrast, President Donald Trump's administration is focused on preserving American supremacy over China, while Anthropic is preparing for a major IPO at an estimated valuation of around $2 trillion in 2026. Trump advisor David Sacks accused the two of merely trying to slow down competitors. "Your goals are not altruistic," he said.
