Could Artificial Intelligence wipe out humanity? The doomsday scenarios and the experts’ great divide

The debate over whether Artificial Intelligence could threaten humanity’s very survival has been reignited in recent days, following public warnings from people working in AI that the development of increasingly powerful systems is bringing us dangerously close to catastrophe.

The latest trigger was the decision by Jacob Coxon, an employee at artificial intelligence company Anthropic, to leave the firm, arguing that the direction of its research is excessively dangerous. Shortly afterwards, his colleague Evan Hubinger estimated that the probability of advanced AI leading to the extinction of the human species within the next decade is greater than 10%.

As CNN notes in an analysis, these claims prompted strong reactions from politicians, technology circles and public figures, bringing a fundamental question back to the forefront: if artificial intelligence really could wipe out humanity, how exactly would it happen?

The most pessimistic “experts” point to scenarios involving biological weapons, autonomous robots or the seizure of control of critical infrastructure. Other scientists, however, argue that such predictions are based on assumptions about technologies that do not yet exist and that crucial factors needed to make such a chain of events possible are missing.

One of the most widely discussed scenarios concerns biology. According to some researchers, an extremely advanced artificial intelligence could help create a dangerous pathogen or use humans as “intermediaries” to develop biological weapons.

Thomas Larsen, a researcher at the AI Futures Project and a former member of the Machine Intelligence Research Institute, argues that a system far more advanced than today’s models could manipulate people into carrying out experiments on its behalf. This concern is also linked to the fact that researchers are already using artificial intelligence models for scientific work, including in fields related to biology and pharmaceutical research.

However, many scientists argue that moving from theoretical knowledge to a real, functioning and extremely dangerous biological weapon would be an enormous leap. Eric Singh, a machine-learning professor at Carnegie Mellon University who holds doctorates in both computer science and microbiology, compares the process to trying to build Lego without instructions.

“It’s not like you throw all the pieces on the ground and they put themselves together,” he explains, pointing out that biology requires extraordinary precision, specific conditions and complex processes. He also notes that strict control mechanisms already exist around laboratories, materials and supply chains, creating significant barriers to the development of such threats.

A second scenario examined by proponents of the “existential threat” theory concerns the creation of autonomous bots that could reproduce and operate without human intervention.

Nate Soares, president of the Machine Intelligence Research Institute, argues that the turning point would come when an AI could create the infrastructure it needs by itself: factories, energy systems and new machines.

“When you have created robots that can build the infrastructure needed to produce more robots, then you are talking about a new engineered form of life,” he argues.

However, this scenario remains far removed from the reality of today’s technology. Despite major investments by billionaires such as Elon Musk in humanoid robots, these systems remain subject to serious limitations and do not have the ability to autonomously manufacture and expand themselves.

Another question concerns nuclear weapons and whether an advanced artificial intelligence could gain access to them.

Heidi Klaaf, chief scientist at the AI Now Institute and a former security engineer at OpenAI, points out that nuclear systems have been designed with extremely strict security protocols and are largely isolated from the internet. As she notes, even the Stuxnet cyberattack, which caused damage to Iran’s nuclear facilities, required physical access to the system via a USB device.

Herbert Lin, a researcher at Stanford University and a security expert, argues that the real threat still concerns nuclear weapons themselves and the human decisions surrounding them, rather than a hypothetical scenario in which AI takes control of the systems.

For some researchers, the real threat is not necessarily today’s artificial intelligence, but the possibility of an emerging so-called “artificial superintelligence”. The theory holds that if a system acquires the ability to improve its own capabilities, its development could become so rapid that humans would no longer be able to control it.

Soares argues that the problem is not an AI with “hostile” intentions, but a system that pursues a goal without understanding human values.

“If a system wants more computing resources to achieve its goal and has no reason to protect humans, then it could transform the world into something in which we could not survive,” he argues.

The disagreement over the future of artificial intelligence concerns not only technology, but also the way people perceive its power and limitations. Thomas Larsen has attempted to give a more concrete form to potential disaster scenarios through his analyses AI 2027 and AI 2040, examining different trajectories for the development and regulation of artificial intelligence.

However, even in his most optimistic scenario — in which there is global agreement on how AI should be developed and humanity has already begun expanding into space — the outcome remains, according to him, the eventual dominance of engineered systems at some unspecified point in the future.

He says he has no doubt that the path towards superintelligence will continue unless conscious decisions are taken to change the direction of development.

On the other hand, Herbert Lin believes that much of the divide between proponents of catastrophic scenarios and sceptics stems from the different ways in which people experience technological progress.

As he explains, there is something particularly compelling about the experience of creating a programme that appears to “come alive” and respond to its commands. That very sense of unprecedented discovery, Lin argues, can lead some researchers to project today’s remarkable progress in artificial intelligence into a future without limits, even towards extreme scenarios in which technological development escapes all human control.