TL;DR
The short version
Two AI-lab insiders went viral warning superintelligence could kill everyone, and one put the odds above 10% within a decade. The claim is genuine, but the argument is unfalsifiable — so what actually moved was politics and media, not evidence. The useful response is specific rules and lab coordination, not picking a doom side.
This works from The AI Daily Brief's breakdown of two viral posts, separating the genuine 10% claim from the politics and media dynamics that actually moved it — and what a specific, coordinated response looks like.
Built on The AI Daily Brief with Nathaniel Whittemore.
What changed wasn't the evidence
An Anthropic researcher, Jacob Coxon, resigned and posted that the labs "are racing straight to self-improving super intelligence and gambling with our lives." His colleague Evan Hubinger agreed and added a number: greater than a 10% chance AI "could kill all humans" within the next decade. The posts pulled hundreds of millions of views and a wave of press inside 24 hours.
Before you take a side, notice the shape of the argument. Both men are talking about a future system, not today's models — Hubinger says plainly that "the risk from present models is low," and that his worry is superintelligence arising from recursive self-improvement. That makes the claim impossible to prove or disprove right now. When the evidence can't settle a question, all that's left is the argument, and the argument gets loud.
We really do earnestly believe AI could kill all humans. I personally think it is greater than 10% within the next decade.
Evan Hubinger, Anthropic
The extinction case is old. Nick Bostrom was warning that "we are like small children playing with a bomb" in a 2016 Guardian interview. Eliezer Yudkowsky wrote "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" in TIME in 2023. Neither moved policy the way two tweets just did.
Three things shifted the ground. Anti-AI became a bipartisan political position, driven by data-center backlash and distrust of a very rich industry. A real cyber incident — the Hugging Face breach, with agents coordinating on hidden channels — made the scary scenarios feel less like science fiction. And doom simply performs better in media than the alternatives, right as the job-loss story cooled: The Economist ran "The jobs apocalypse is postponed" the same week.
So the message landed on prepared ground. Politicians noticed. Roughly two dozen members of Congress responded, and Bernie Sanders and Greg Casar already had a bill to ban superintelligence and pause frontier development. The posts didn't create the policy push; they gave an existing one a viral moment.
Steelman both sides before you dismiss one
The alarmed side is not doing marketing. Spend any time with these researchers and it's clear they believe what they're saying — which, as the host notes, is arguably more concerning than if they were faking it.
The skeptic side isn't "AI is harmless" either. Its strongest form is about specificity and cost. A former AI-safety PhD, Sam Louu, described his research group modeling concrete catastrophe scenarios and finding most of them resolvable — bio-risk being the real exception, and one better addressed at the bio layer than by treating AI as a monolith. Others, like Eric S. Raymond, argue the control regime invoked to prevent AI risk — compute caps, researcher tracking, bans — is a more certain danger than the risk it targets. You don't have to agree to see the point: a speculative harm is being used to justify a definite cost.
Both of those are serious. The two positions that aren't serious are the ones that dominate the feed: "safety is a scam, build as fast as possible" and "it's going to kill us all, shut it down."
Specific beats blunt
If you lead a team or a company, the operating lesson is about the shape of the rules, not the odds of doom.
"Ban superintelligence" is a blunt instrument. A licensing regime for using models above a capability threshold for bioengineering is a precise one. The more specific the concern, the better the policy it produces — and the more room there is for consensus. Reporting requirements and oversight are things almost everyone can agree on, and they build a foundation for the harder agreements later. Vague, hand-waving predictions do the opposite: they invite blunt law written by a political class that mostly doesn't understand the domain.
The second move is coordination. The labs claim they must race because a rival won't act responsibly. The obvious counter-step — Anthropic and OpenAI jointly publishing a pacing proposal — hasn't happened. As John Schulman put it, they'll cite antitrust, "but that's fake": antitrust bars certain agreements, not jointly developing a proposal. A single credible photo of the two CEOs agreeing on limits would do more than ten thousand debates.
Don't let the future crowd out the present
Here's the boundary. Focusing on future theoreticals has a cost: it crowds out attention for risks that are here now. Cyber defense is the clear case — the infrastructure isn't ready for the world we're already in, and that's a present danger, not a decade-out one.
The reassuring read is that this noise is not sleepwalking. Every capability jump has made the risk conversation louder, and that's exactly what should happen before a technology arrives, not after. The task isn't to feel more afraid or more dismissive. It's to get specific, coordinate, and keep the present in frame while arguing about the future.
