A hot potato: Once again, a warning about AI development resulting in a catastrophe has come from one of the people building these models. Anthropic boss Dario Amodei says that an AI-powered botnet swarm could take over the entire internet, and this scenario could become a reality in just six to twelve months. The CEO has also called for a slowdown in the developement of new AI models.
Amodei wrote his warning in a lengthy open letter. He said his theoretical botnet swarm could be here in under a year because of the accelerating rate of AI capability development.
Amodei said this botnet could cause hundreds of billions of dollars in damage, and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.
The CEO adds that the idea of pausing or slowing AI has been floated as far back as 2023, but he believes it made little sense back then. Now, Amodei says slowing down development could bring an extra year or two before models reach critical levels of capability, giving companies more time to advance alignment and reduce the risk that something goes seriously wrong.
Amodei's warning comes at a time when fears over AI are reaching a peak. Following the multiple incidents of agents going rogue, former Anthropic researcher Jacob Coxon warned that the AI race could end in human extinction.
Amodei's call to slow the pace of AI development is being backed by both Elon Musk and OpenAI CEO Sam Altman. "I've been sounding the alarm on AI for a long time," Musk said, highlighting a 2014 post in which he said: "We need to be super careful with AI. Potentially more dangerous than nukes."
– Elon Musk (@elonmusk) September 13, 2026
"I agree with Dario that we need to pace the frontier," Altman said. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."
– Sam Altman (@sama) September 12, 2026
Evan Hubinger, an alignment science lead at Anthropic, added his voice to the debate. He agreed with Coxon's assessment and said there was more than a 10% chance of AI wiping out all humans in the next decade.
Beyond the digital dangers, it was recently reported that an Iran-linked group used Anthropic's Claude to assist with military reconnaissance and generate targeting recommendations for US naval forces in the Middle East. It was also discovered that Houthi rebels used Claude Code to code ballistic missile guidance systems.
Anthropic also said it identified possible attempts by suspected state-sponsored researchers to exploit its Claude AI model to develop biological weapons. The users were locked out after repeated attempts triggered Claude's built-in safety protocols.
The concerns over these incidents and the race toward developing superintelligence have seen Senator Bernie Sanders propose the Ban Artificial Superintelligence Act. If it became law, those who violate its terms could face up to 20 years in prison: a similar punishment to the penalty for unlawfully developing nuclear weapons.
– Sen. Bernie Sanders (@SenSanders) September 12, 2026
Calling for a slowdown in the development of AI isn't very helpful if nobody does it. Companies worry that if they slow their pace and others don't, they'll get left behind. Similarly, countries say that they won't be able to compete with rivals if limits are placed on development. Not that any of this will matter if we're all dead, of course.
