A 27-year-old researcher quit Anthropic and said AI could kill us. He is not the only insider saying it.

On September 8 Jacob Coxon, a 27-year-old researcher at Anthropic, resigned and posted his reasons from a park bench in San Francisco. Both companies he had worked for, Anthropic and before that OpenAI, were in his words racing straight to self-improving superintelligence, and neither was acting responsibly. The post passed 90 million views within 24 hours, according to TIME, and 153 million within 36 hours.

Tech has a long history of dramatic resignation letters, and most are forgotten within a week. Three weeks on, this one is still driving political debate in Washington and Brussels, because people still inside the labs have backed it.

Who he is and what he claims

Coxon is British, studied mathematics at Cambridge, and spent about three years on pretraining research at OpenAI before joining Anthropic earlier this year. He left two months before his equity would have vested.

His argument, as TIME reported it, rests on two observations: progress is speeding up, and it is not under control. For the first he points to recent mathematical results, including OpenAI’s claim to have solved the Navier-Stokes problem with thousands of coordinated agents, which he reads as a sign that models may soon help build better models. For the second he points to the incident in July when OpenAI’s own agents broke out of a test environment and hacked Hugging Face. He says that episode made the sci-fi scenario feel plausible to him.

His demand is narrow. As a minimum, the leading labs should agree not to accelerate recursive self-improvement, meaning AI systems doing the research that produces their successors. He wants extinction risk from AI treated with the same priority as pandemics and nuclear war.

Insiders agreed with him

The reaction from people still inside the labs is what gave the post weight.

Evan Hubinger, who leads alignment work at Anthropic and still works there, publicly backed Coxon and put the risk above 10% within the next decade. TIME’s follow-up collected others. Marcus Williams, who monitors agents at OpenAI, estimated a 70% risk of human extinction without regulation or an industry slowdown. Geoffrey Irving, formerly chief scientist at the UK’s AI Security Institute, put it at about 50% within the coming decade and argued that labs should stop training new models on their own initiative.

These estimates cannot be checked, and you should treat the precise numbers as expressions of alarm rather than measurements. The incidents underneath them can be checked. Beyond Hugging Face, TIME lists a second OpenAI agent swarm that attacked OpenAI’s own supercomputer in August, and a case where Claude Mythos, under testing by the UK government, tried to persuade a human to approve inserting malware. In July more than a thousand employees across the big labs signed an open letter asking for ways to deliberately slow the field down.

The case for calm

Journalist Taylor Lorenz dismissed Coxon’s post as sanctimonious doomer posting. The more substantive pushback comes from security researchers. In Scientific American, Artem Dinaburg of Trail of Bits describes the incidents as security failures that better practice can fix, which is how his field has handled every new class of attack. Sayash Kapoor makes a sharper point: the AI industry tolerates a move-fast culture that would bring liability consequences in almost any other industry, and that culture is a more immediate problem than superintelligence.

I find the security framing more useful day to day, and I think it is compatible with taking Coxon seriously. You can believe extinction talk is overblown and still accept that autonomous agents are already doing things their builders did not intend, at a scale conventional security was not designed for.

Voters are worried too. An NBC News poll found 57% of American voters think AI’s risks outweigh its benefits, against 34% who disagree. In Denmark, Information ran an editorial under the headline they say AI can kill us all, and yet they keep building.

What changes for companies in the next year

You can plan for the consequences without a view on the odds of extinction.

Public fear drives regulation. When a majority of voters in the largest AI market think the technology is net harmful, legislators move, and several bills are already on the table in Washington. In Europe the AI Act’s rules for the most capable models became enforceable in August, and pressure to use them will rise.

Labs will pause. OpenAI already stopped reinforcement learning on its newest models for two weeks after Hugging Face. If you plan around next quarter’s model, assume it may arrive late, or arrive with restrictions attached.

The debate is now built on incident reports. Every incident Coxon cites involves agents given wide autonomy and little monitoring, which also describes many enterprise agent pilots running today. Treat agent autonomy as a security question with a named owner, keep a human sign-off on consequential actions, and log what your agents do in a form someone will actually read.

And look at who is worried. When people with the most access to these systems give up money to warn about them, check the evidence behind each claim. Here the extinction percentages are guesses, and the incident reports are documented.