In the space of one week, the people building artificial intelligence told the world it could end all of humanity. This month, Evan Hubinger, lead of Alignment Science at Anthropic, put in his own odds, “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” He also made a distinction: the risk from today’s models is low.
Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, posted that “The people building AI earnestly believe that it could kill us all by the end of the decade.” He resigned that day, saying the labs were “gambling with our lives.”
Anthropic chief executive, Dario Amodei, published a 4,000-word essay warning that within six to 12 months a swarm of AI agents “could be capable of taking over the entire internet with a persistent botnet,” and calling on the industry to slow down. Sam Altman agreed. Elon Musk wrote, “Dario is right.” Fierce competitors were suddenly aligned and asking for government intervention. Vice President JD Vance said that he suspects a Trojan horse.
All of this plays on our basic survival instinct, a fear of the unknown. Part of the fear may be generating from a programming concern of unpredictability. AI has surpassed expectations and does so at a speed that is becoming increasingly difficult to contain. But, human extinction? Not a chance.
P(doom)
In the tech world, this anxiety is called P(doom), the probability of doom. It has become a dinner-party number, a badge of ethical honor, and increasingly, a political instrument.
Heidy Khlaaf, chief scientist at the AI Now Institute, put it plainly this month: “These claims are neither falsifiable nor verifiable, making them unscientific by their very nature.” A claim that cannot be tested cannot be disproved, and a claim that cannot be disproved is a remarkably useful thing to carry into a Senate hearing.
That deserves a closer look. Amodei’s essay asks for three things: independent evaluators embedded within the labs (they just so happen to know of some), a coordinated slowdown among Western companies (competition beware), and, eventually, a global agreement including China (getting rid of cheap AI competition).
The second requires something extraordinary: an antitrust waiver from Washington, so that these tech leaders may sit in a room and agree on how fast to build. Dirk Auer, a legal scholar, has already called it a “crisis cartel.” Steven Sinofsky, the former Windows chief, made the sharper point: if Anthropic wants to slow down, it literally just takes an email from the founder-CEO. You do not need the government’s permission to take your own foot off the gas. You need the government’s permission to take everyone else’s. And, therein lies the Trojan horse.
Furthermore, if Western tech leaders form a “crisis cartel” to coordinate a slowdown, they do not merely pause innovation; they actively cede the global security architecture to state-sponsored actors. Voluntarily throttling our development opens a geopolitical security vacuum, handing adversarial nations a critical advantage in the next era of warfare and economic dominance.
However, let’s be fair because the fear is not fake. More than 1,300 employees at frontier labs signed a letter in July asking Washington for tools to throttle automated AI research. Engineers do not petition to create obstacles to their work.
The alarm is sincere, it is the proposed cure that serves existing leaders … licensing thresholds that only the largest can afford, liability shields for those already inside, and a speed limit written by the two companies at the front of the pack. Sincerity and self-interest are not mutually exclusive; in Washington, they are roommates.
Nvidia Chief Executive Jensen Huang, whose chips power all of it, offered the cynic’s version: “What better way to create demand than to create a problem?” And, “Who doesn’t want their market to be hysterical about their product and line up around the corner for it?” As those in Washington say, never let a crisis go to waste. And … this is midterm candy.
The wrong solution
Panic about new technology is nothing new. Socrates warned that writing would destroy memory. We feared electricity, the automobile, nuclear power, the internet, cellphones … each time we adapted, and each time the adaptation made us more capable, not less. The fear was never the technology, it was the speed, the uncertainty that we might not keep up, thus lose control.
AI ISN’T BECOMING SENTIENT — IT’S BECOMING YOU
That instinct is human, and it is survival. But instinct is not science. A doomsday frame leads to one prescription: slow everything, license everything, and conveniently hand the keys to whoever is already inside. Forget that you cannot regulate what you don’t understand; there is a likelihood of hidden repercussions.
The real risk is not a machine that suddenly goes power crazy; it is a machine that quietly rewrites itself faster than any human can read, until the people in charge are watching outputs they can no longer trace. In my op-ed on why AI self-improvement is a Recursive Hallucination, I lay out how the fascination with AI self-programming can lead to an inability of humans to detect flaws and correct them, which is a genuine security risk.
Jacqueline Cartier is a corporate and legislative strategist focused on communications, crisis leadership, public trust, and emerging technologies that shape human behavior and decision-making. Follow her on LinkedIn.