AI pioneer warns against race for superintelligence

Nobel laureate Geoffrey Hinton has warned against rushing to develop artificial superintelligence, arguing that scientists still do not know how to ensure systems more capable than humans remain safe and under control.

Hinton, a British-Canadian computer scientist whose pioneering work helped lay the foundations for modern artificial intelligence, has become one of the technology's most prominent critics. He shared the 2024 Nobel Prize in Physics for work underpinning machine learning with artificial neural networks.

His latest warning centres on the prospect of superintelligence - AI capable of surpassing humans across a broad range of intellectual tasks.

Hinton told The Times that pursuing such systems without first establishing how they could be safely controlled would expose humanity to potentially severe consequences.

“We would be very foolish to start developing superintelligence now, when there is no scientific consensus that it can be developed safely and kept under control,” Hinton told The Times.

His concern is not simply that increasingly powerful AI could make mistakes. Hinton has warned that a system more intelligent than humans could become difficult or impossible to control, potentially posing a threat to humanity itself.

Hinton has previously estimated a 10-20 per cent chance that AI could lead to human extinction within the next 30 years, citing the unexpectedly rapid pace of technological development.

A central concern, he argues, is that humanity has no previous experience of dealing with entities more intelligent than itself. That makes the emergence of machines exceeding human cognitive abilities fundamentally different from earlier technological advances.

Hinton has also challenged the assumption that today's advanced AI systems merely reproduce patterns found in their training data. He believes sophisticated language models can grasp semantic and abstract concepts, a development that has contributed to his growing concern about their capabilities.

Rather than relying on technology companies to police themselves, Hinton has called for governments and regulators to play a stronger role in determining how advanced AI develops.

He has compared technological innovation with the accelerator in a car, while describing government regulation as the steering mechanism needed to determine where that vehicle goes.

Hinton has also rejected the idea that market forces alone will adequately address the dangers. Commercial competition encourages companies to build increasingly capable systems, while fears of falling behind international rivals can make businesses reluctant to accept restrictions, he argues.

That problem is intensified by geopolitical competition, including the technological rivalry between China and Western countries. Companies may fear that slowing development to introduce safeguards would hand an advantage to competitors operating under different rules.

Hinton spent years at Google and played a major role in research that helped drive the rapid expansion of modern machine learning. He left the company in 2023, saying the move allowed him to speak more freely about the risks posed by AI.

His warnings carry particular weight because of his contribution to the technology he now fears could become dangerous. In 2024, Hinton and John Hopfield were awarded the Nobel Prize in Physics for discoveries and inventions that enabled machine learning with artificial neural networks.

Hinton maintains that humanity still has an opportunity to influence how increasingly powerful AI is developed and governed. His warning, however, is that the opportunity to establish effective safeguards may narrow as companies race towards systems whose capabilities exceed our own.