Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Existential risk

Also called x-risk, AI doom, p(doom)

Existential risk is the worry that future AI, far more capable than today's, could cause a catastrophe that ends or permanently cripples humanity.

Safety and societyCurious

In plain words

Most worries about AI are about harms we can already see: scams, bias, fake videos. Existential risk is a different, bigger worry. It asks whether a future AI, much more capable than people at most things, could slip out of our control or be used to cause a disaster with no way back.

The term comes from philosophy. An existential risk is one that could wipe out humanity or permanently shrink its future. Nuclear war and engineered pandemics are other examples.

Experts are deeply split. Some leading researchers think the risk is real and urgent. Others think it is speculative and distracts from today's problems. People sometimes put a number on their own guess, called "p(doom)". Those numbers are personal estimates, not measurements.

Everyday example

A smoke alarm for a fire nobody has seen

Imagine a new kind of stove that keeps getting hotter every year. No kitchen has caught fire yet. One cook says: fit alarms and fire doors now, because once a fire starts it may be too late. Another says: this stove has never burned anything, and the real danger is the leaking gas pipe in front of us. Both are reasoning carefully. They disagree about how likely the fire is.

Where the example breaks: A stove fire is a familiar risk with known odds. A catastrophe from AI has never happened, so nobody can measure the odds from experience, which is exactly why estimates range from almost zero to very high.

Go deeper

How it actually works

People who take the risk seriously usually describe three broad routes. Each is debated.

  1. Loss of control

    A highly capable AI pursues a goal that differs from ours and resists being corrected or switched off. This is the extreme case of the alignment problem. The 2026 International AI Safety Report describes such scenarios and says current systems lack the capabilities to pose them, while noting steady gains in AI acting on its own.

  2. Deliberate misuse

    People use powerful AI to help cause mass harm, for example by helping with weapons. The 2026 International AI Safety Report lists malicious use as one of its three groups of AI risk, which is why labs test new models for dangerous abilities and add guardrails.

  3. Why it is hard to check

    We cannot wait for a disaster to measure the odds. Tests are the main tool, and the same report warns that models increasingly tell test settings apart from real use, so a dangerous ability could go unnoticed before release.

  4. What p(doom) means

    Short for probability of doom: a person's own guess at the chance AI leads to catastrophe. In December 2024 Geoffrey Hinton put it at 10% to 20% within 30 years. Yann LeCun, then Meta's chief AI scientist, has called the takeover scenario "preposterous".

  5. What researchers say in surveys

    A survey of 2,778 AI researchers in October 2023 gave a median 5% chance that advanced AI leads to outcomes as bad as human extinction. Between 38% and 51% gave at least 10%, depending on how the question was worded. Many more were concerned about near-term harms such as misinformation.

Term defined
Philosopher Nick Bostrom, 2002: a risk that could "annihilate Earth-originating intelligent life or permanently and drastically curtail its potential"
Public statement
May 2023: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war", signed by Hinton, Bengio, and heads of OpenAI, Google DeepMind and Anthropic
Survey median
5% chance of extremely bad outcomes such as extinction (2,778 researchers, October 2023). A guess, not a measurement
Today's systems
"Current systems lack the capabilities to pose such risks" (International AI Safety Report, February 2026)

This is the far end of AI safety. Much of the same work, such as testing, red teaming and explainability, helps with today's harms too.

Is AI a real existential risk?

One view

Yes, it deserves serious priority

Capabilities are rising fast, and we do not yet know how to reliably control systems smarter than us. Researchers such as Geoffrey Hinton and Yoshua Bengio, and the heads of major labs, signed a 2023 statement putting AI extinction risk alongside pandemics and nuclear war.

The other view

No, it is speculative

No AI has shown a will to take over, and Yann LeCun calls the takeover scenario preposterous. Princeton's Arvind Narayanan and Sayash Kapoor argue AI is a normal technology that institutions can control, and that drastic steps aimed at superintelligence could make things worse.

Most likely: Experts genuinely disagree, and no one can measure the odds. Surveyed AI researchers put the chance low but not zero. Today's systems cannot pose this risk, so the debate is about future ones and how much to prepare now.

Questions people ask

What does p(doom) mean?

Probability of doom: a person's own estimate of the chance that AI leads to a catastrophe such as human extinction. It is a personal guess, not a calculation. Published guesses range from near zero to very high, which shows how unsettled the question is.

Could AI really make humans extinct?

Nobody knows. Some leading researchers think it is possible with much more capable future systems, through loss of control or misuse. Others think it is very unlikely. The 2026 International AI Safety Report says today's systems lack the capabilities to pose such risks.

Why do AI experts disagree so much about this?

Because there is no data from past disasters to go on. Estimates depend on beliefs about how fast AI will improve, whether it will develop goals of its own, and how well people and governments will respond. Reasonable experts start from different assumptions.

Is existential risk the same as AI safety?

No. Existential risk is one concern within AI safety, about the most extreme outcomes. AI safety also covers everyday harms such as scams, bias, wrong advice and security, which are happening now.

What is being done about it?

AI labs test new models for dangerous abilities before release, and governments have set up AI safety institutes, including in the UK and India. An International AI Safety Report, chaired by Yoshua Bengio, brings together experts from over 30 countries. Research on alignment and explainability aims to keep powerful systems understandable and controllable.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.