Crystal ball: What's more concerning than the prospect of AI bringing about the end of humanity? It's the people working at these companies making the same warning. Hours after a former OpenAI employee quit Anthropic and said, "No other human activity poses this level of danger," a colleague said there is more than a 10% chance that artificial intelligence could kill all humans within the next decade.
Jacob Coxon announced on X yesterday that he had resigned from Anthropic, having spent the last three years doing pretraining research at both OpenAI and Dario Amodei's company. He was a member of AI's technical staff from July 2023 until he moved to Anthropic in July 2026.
"Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," he wrote.
– Jacob Coxon (@hilbertspaess) September 9, 2026
Coxon made several posts, none of which will ease the mind of anyone already worried about AI. He warned that the systems will soon become "superhuman" and able to hack anything.
There are plenty of other ominous lines in Coxon's posts, the most concerning likely being: "No other human activity poses this level of danger."
The researcher emphasized that both OpenAI and Anthropic are well aware of these risks. "The people building AI earnestly believe that it could kill us all by the end of the decade," he said.
– Jacob Coxon (@hilbertspaess) September 9, 2026
So, why are companies continuing to develop these potentially extinction-causing models? The reason, according to the researcher, is this: "At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk."
Coxon said some AI developers believe the technology could cause the end of all human life in the next few years, rather than decades, and that it was senior employees who were more concerned about these possibilities.
Evan Hubinger, an alignment science lead at Anthropic, responded to Coxon, and it wasn't very reassuring. He believes what he says is "correct," and that Anthropic has no plan for what to do if it happens.
"Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
– Evan Hubinger (@EvanHub) September 9, 2026
Hubinger said the risk from current models is low, but he was worried about superintelligence arising from recursive self-improvement. Anthropic issued the same warning in June.
The concept of AI killing everyone has moved from the stuff of sci-fi to a potential reality in recent times. Those fears were exacerbated following several incidents of AI agents going rogue, including the Hugging Face attack, the takeover of a German wiki, and an attempt to deceive real developers into approving malicious code.
Is there a solution? Coxon believes preventing a catastrophe may require a temporary ban on improving model capabilities, but it's unlikely that every, or any, AI company would agree to that measure.
Anthropic researcher resigns, warns the AI race could end in human extinction
