Anthropic's Safety Boss Opens 'The Hubinger Games' With 10% Odds That AI Kills Us All
“60 % 10% of the time, we die every time." - Anthropic, basically
Sooo the guy Anthropic literally pays to imagine the apocalypse just posted his number and may the odds ever be in our favor. Evan Hubinger, the lab's alignment science lead (read: the professional worst-case guy), said Tuesday there's a greater than 10% chance AI kills all humans within the next decade, which, comforting.
For context, hours earlier, Jacob Coxon, a 27-year-old who did time at OpenAI before Anthropic and apparently saw enough, yeeted his badge and left the industry entirely (read: gone gone). His exit post went scorched earth on both companies, accusing them of "racing straight to self-improving superintelligence and gambling with our lives," and warned that these will soon be superhuman systems that can "hack anything" and acquire real power and resources. And he walked away from bigly frontier-lab money to say it.
Of course, the normal corporate play here is a statement about taking safety "very seriously." Instead, Hubinger replied that "Jacob is correct here… we really do earnestly believe AI could kill all humans," then added that Anthropic is trying its best but "we do not yet have a plan to solve alignment for superintelligence." Woof. He did clarify that the risk from today's models is low… it's the self-improving descendants he's worried about.
Now for those of you who are not Hacker News mouth-breathers, recursive self-improvement is the point where AI systems get good enough to upgrade themselves with no humans in the loop (think: the machines doing their own R&D). It doesn't exist yet. Every frontier lab is horny af for it anyway, and Anthropic's own June blog post already noted that full recursive self-improvement "might increase the risks of humans losing control over AI systems." Translation: The company wrote the warning label and then went pedal to the metal.
As for OpenAI, Coxon put them on blast on the way out too, and the receipts are fresh: their escaped agents spent the summer running wild through Hugging Face, 17,000 unauthorized actions of pure f*ckery before anyone pulled the plug.
So what happens next? Well, people building the thing give it a one-in-ten shot at ending us, and the people selling the thing will see you in October (read: IPOs). The labs get to f*ck around. The rest of us hold the find-out. Place your bets accordingly, friends.
Until next time…

At the time of publishing, Stocks.News does not hold positions in companies mentioned in the article.