OpenAI chief scientist urges ‘extreme caution’ with pace of artificial intelligence

OpenAI’s top scientist has warned that artificial intelligence is evolving so rapidly that it is becoming increasingly difficult for humans to understand and control, and said he expects AI labs to voluntarily slow development for safety reasons.

In a relatively short space of time, AI models can operate computers, collaborate with humans and other AI systems and carry out research projects, wrote chief scientist Jakub Pachocki in a post on Sunday. It won’t be long until they can also improve themselves without human intervention, a process known as recursive self-improvement, he said.

“This is a time that calls for extreme caution,” Pachocki wrote in the post. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.”

Developers can either try and steer AI more closely to align with human interests or “slow down future development as needed,” he wrote. “I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.”

Pachocki’s remarks are consistent with broader doomsaying by AI technology executives. Anthropic PBC Chief Executive Officer Dario Amodei warned that AI could replace half of all entry-level jobs. OpenAI CEO Sam Altman once made a similar prediction, but told Bloomberg TV in an interview last week that the AI industry hadn’t done enough to communicate the potential benefits to humanity.

Still, the giddy pace of AI releases has coincided with breakthroughs in cybersecurity capabilities, scientific discoveries and apparent formalization of longstanding mathematical problems. OpenAI has opted for a limited release of its latest GPT-6 Astra model, citing its advanced cybersecurity capabilities. Anthropic has not yet released its cutting-edge Mythos model to the public for similar reasons.

A growing number of startups are trying to bring about the very thing Pachocki warns about — self-improving AI. Investors are flooding into recursive self-improvement, betting that this is the route to artificial intelligence that is truly human-like, or more impressive. Inherent, founded by former DeepMind researchers, raised $50 million earlier this year. Recursive Superintelligence, an AI lab from renowned scientist Richard Socher pursuing similar research, also landed $650 million in financing.

Pachocki’s employer, OpenAI, has said it wants to build a “true automated AI researcher” in less than two years.

Discover the latest Business News, Sensex, and Nifty updates. Obtain Personal Finance insights, tax queries, and expert opinions on Moneycontrol or download the Moneycontrol App to stay updated!

Find the best of Al News in one place, specially curated for you every weekend.

Stay on top of the latest tech trends and biggest startup news.