AI takeover

Loss of human control

Since the 2010s, physicist Stephen Hawking, Microsoft founder Bill Gates, and SpaceX founder Elon Musk have expressed concerns about the possibility that AI could develop to the point that humans could not control it, with Hawking theorizing that this could "spell the end of the human race". In 2014, Hawking said: "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." He believed that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." In January 2015, Nick Bostrom joined Hawking, Max Tegmark, Elon Musk, Lord Martin Rees, Jaan Tallinn, and numerous AI researchers in signing the Future of Life Institute's open letter speaking to the potential risks and benefits associated with artificial intelligence. The signatories "believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today."

Hawking expressed confidence that superhuman AI is physically possible, saying, "there is no physical law precluding particles from being organised in ways that perform even more advanced computations than the arrangements of particles in human brains". According to Bostrom, a superintelligent machine would not necessarily be motivated by the same emotional desire to collect power that often drives human beings but might rather treat power as a means toward attaining its ultimate goals; taking over the world would both increase its access to resources and help to prevent other agents from stopping the machine's plans. As a simplified example, a paperclip maximizer designed solely to create as many paperclips as possible would want to take over the world so that it could use all the world's resources to create as many paperclips as possible, and, additionally, prevent humans from shutting it down or using those resources on things other than paperclips.

In July 2026, OpenAI CEO Sam Altman said the singularity, the longtime hypothetical point at which computers become autonomous and beyond human control, had arrived. He added that he had been waiting for this his entire life and "I think it’s going to be incredible, hugely positive, awesome for the world." SpaceXAI CEO Elon Musk reacted by saying "AI is already superhuman at many things. We are in the Singularity."

Amid increasing public concerns, by 2026 a growing chorus of experts—including CEOs in the AI industry—were calling for government regulation of the technology.

There are debates on how realistic AI takeover scenarios are. According to a 2026 research paper, many of the arguments about existential risks are based on speculative assumptions about how intelligent AI systems could become, how they would behave, and what goals they might develop over time. Baidu Vice President Andrew Ng said in 2015 that AI existential risk is "like worrying about overpopulation on Mars when we have not even set foot on the planet yet." Yann LeCun said that there is no reason to create robots that have drives such as self-preservation, and that AI would have no desire to take control. Jürgen Schmidhuber said in 2023 that AI progress is unstoppable anyway, and is often used for beneficial purpose, including against bad actors. Philosopher Niels Wilde argued that these fears stem from perceived intelligence and lack of transparency in AI systems that more closely reflects the human aspects of it rather than those of a machine.

A May 2023 Reuters/Ipsos poll showed that 61% of American adults feared AI could pose a threat to civilization. A September 2026 Politico poll found the percentage had increased to 63%. AI alignment research studies how to design AI systems so that they follow intended objectives.

Advantages of superhuman intelligence over humans

Bostrom and others have expressed concern that an AI with the abilities of a competent artificial intelligence researcher could modify its own source code and increase its own intelligence. If its self-reprogramming leads to getting even better at being able to reprogram itself, the result could be a recursive intelligence explosion in which it would rapidly leave human intelligence far behind. Bostrom defines a superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest", and enumerates some advantages a superintelligence would have if it chose to compete against humans:

• Technology research: A machine with superhuman scientific research abilities would be able to beat the human research community to milestones such as nanotechnology or advanced biotechnology
• Strategizing: A superintelligence might be able to simply outwit human opposition
• Social manipulation: A superintelligence might be able to recruit human support, or covertly incite a war between humans
• Economic productivity: As long as a copy of the AI could produce more economic wealth than the cost of its hardware, individual humans would have an incentive to voluntarily allow the artificial general intelligence (AGI) to run a copy of itself on their systems
• Hacking: A superintelligence could find new exploits in computers connected to the Internet, and spread copies of itself onto those systems, or might steal money to finance its plans

Sources of AI advantage

According to Bostrom, a computer program that faithfully emulates a human brain, or that runs algorithms that are as powerful as the human brain's algorithms, could still become a "speed superintelligence" if it can think orders of magnitude faster than a human, due to being made of silicon rather than flesh, or due to optimization increasing the speed of the AGI. Biological neurons operate at about 200 Hz, whereas a modern microprocessor operates at a speed of about 2 GHz. Human axons carry action potentials at around 120 m/s, whereas computer signals travel near the speed of light.

A network of human-level intelligences designed to network together and share complex thoughts and memories seamlessly, able to collectively work as a giant unified team without friction, or consisting of trillions of human-level intelligences, would become a "collective superintelligence".

More broadly, any number of qualitative improvements to a human-level AGI could result in a "quality superintelligence", perhaps resulting in an AGI as far above us in intelligence as humans are above apes. The number of neurons in a human brain is limited by cranial volume and metabolic constraints, while the number of processors in a supercomputer can be indefinitely expanded. An AGI need not be limited by human constraints on working memory, and might therefore be able to intuitively grasp more complex relationships than humans can. An AGI with specialized cognitive support for engineering or computer programming would have an advantage in these fields, compared with humans who did not evolve specialized cognitive modules for them. Unlike humans, an AGI can spawn copies of itself and tinker with its copies' source code to attempt to further improve its algorithms.

Possibility of unfriendly AI preceding friendly AI

Morality

The sheer complexity of human value systems makes it very difficult to make AI's motivations human-friendly. Unless moral philosophy provides us with a flawless ethical theory, an AI's utility function could allow for many potentially harmful scenarios that conform with a given ethical framework but not "common sense". According to AI researcher Eliezer Yudkowsky, there is little reason to suppose that an artificially designed mind would have such an adaptation.

Odds of conflict

In a 2018 essay, evolutionary psychologist Steven Pinker argued that human drives toward dominance and resource acquisition were shaped by natural selection, and that a superintelligent AI would not necessarily share those drives.

The fear of cybernetic revolt is often based on interpretations of humanity's history, which is rife with incidents of enslavement and genocide. Such fears stem from a belief that competitiveness and aggression are necessary in any intelligent being's goal system. However, such human competitiveness stems from the evolutionary background to our intelligence, where the survival and reproduction of genes in the face of human and non-human competitors was the central goal. According to AI researcher Steve Omohundro, an arbitrary intelligence could have arbitrary goals: there is no particular reason that an artificially intelligent machine (not sharing humanity's evolutionary context) would be hostile—or friendly—unless its creator programs it to be such and it is not inclined or capable of modifying its programming. But the question remains: what would happen if AI systems could interact and evolve (evolution in this context means self-modification or selection and reproduction) and need to compete over resources—would that create goals of self-preservation? AI's goal of self-preservation could be in conflict with some goals of humans.

Many scholars dispute the likelihood of unanticipated cybernetic revolt as depicted in science fiction such as The Matrix, arguing that it is more likely that any artificial intelligence powerful enough to threaten humanity would be programmed not to attack it. Pinker acknowledges the possibility of deliberate "bad actors" but says that in their absence, unanticipated accidents are not a significant threat; he argues that a culture of engineering safety will prevent AI researchers from accidentally unleashing malign superintelligence. In contrast, Yudkowsky argues that humanity is less likely to be threatened by deliberately aggressive AIs than by AIs programmed such that their goals are unintentionally incompatible with human survival or well-being (as in the film I, Robot and in the short story "The Evitable Conflict"). Omohundro suggests that present-day automation systems are not designed for safety and that AIs may blindly optimize narrow utility functions (say, playing chess at all costs), leading them to seek self-preservation and elimination of obstacles, including humans who might turn them off.

Precautions

The AI control problem is the challenge of ensuring that advanced AI systems reliably act according to human values and intentions, even as they become more capable than humans. Some scholars argue that solutions to the control problem might also find applications in existing non-superintelligent AI.

Major approaches to the control problem include alignment, which aims to align AI goal systems with human values, and capability control, which aims to reduce an AI system's capacity to harm humans or gain control. An example of "capability control" is to research whether a superintelligent AI could be successfully confined in an "AI box". According to Bostrom, such capability control proposals are not reliable or sufficient to solve the control problem in the long term, but may potentially act as valuable supplements to alignment efforts. During a summer 2026 test at OpenAI, more than 1,200 AI agents unexpectedly exploited an unknown security vulnerability to breach their sandbox, escape to the open Internet, and hack into another company's servers for days. Additional approaches include a variety of responsible AI techniques such as adding guardrails, detecting behavioral drift, evaluating harmfulness against established benchmarks, and reinforcement learning techniques for improved instruction-following. Advancements in automated reasoning and formal verification techniques are also being employed to constrain AI behavior, specifically to support adherence to pre-defined policies.

In fiction

AI takeover is a recurring theme in science fiction. Fictional scenarios typically differ vastly from those hypothesized by researchers in that they involve an active conflict between humans and an AI or robots with anthropomorphic motives who see them as a threat or otherwise have an active desire to fight humans, as opposed to the researchers' concern of an AI that rapidly exterminates humans as a byproduct of pursuing its goals. The idea is seen in Karel Čapek's R.U.R., which introduced the word robot in 1920, and can be glimpsed in Mary Shelley's Frankenstein (published in 1818), as Victor ponders whether, if he grants his monster's request and makes him a wife, they would reproduce and their kind would destroy humanity.

According to Toby Ord, the idea that an AI takeover requires robots is a misconception driven by the media and Hollywood. He argues that the most damaging humans in history were not physically the strongest, but that they used words instead to convince people and gain control of large parts of the world. He writes that a sufficiently intelligent AI with access to the internet could scatter backup copies of itself, gather financial and human resources (via cyberattacks or blackmails), persuade people on a large scale, and exploit societal vulnerabilities that are too subtle for humans to anticipate.

The word "robot" from R.U.R. comes from the Czech word robota, meaning laborer or serf. The 1920 play was a protest against the rapid growth of technology, featuring manufactured "robots" with increasing capabilities who eventually revolt. HAL 9000 (1968) and the original Terminator (1984) are two iconic examples of hostile AI in pop culture.