Anthropic and OpenAI Scientists Urge Policymakers to Embed Auditors Inside AI Labs
Anthropic and OpenAI Scientists Urge Policymakers to Embed Auditors Inside AI Labs
Senior scientists from Anthropic, OpenAI and Microsoft are calling on policymakers to embed auditors in frontier artificial intelligence companies and set “concrete safety requirements” for continued development of the technology.
Anthropic Co-Founder Jack Clark, OpenAI Chief Scientist Jaku Pachocki and Microsoft Chief Scientific Officer Eric Horvitz are among 22 senior scientists of frontier AI companies, academics and independent experts from civil society who are calling on policymakers to “urgently prioritize” these and other measures for AI safety.
Their recommendations were included in a paper published Monday (Sept. 28) by the University of Cambridge.
The paper focuses on possible risks caused by an “intelligence explosion” as AI plays a greater role in developing AI, thereby accelerating the technology’s progress and potentially outpacing the ability of humans to oversee it, according to the paper’s executive summary.
For example, the share of internal AI research and development work completed by Anthropic’s AI systems with only high-level supervision rose from 1% to 26% during the five-month period that ended in August, per the summary.
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
“An intelligence explosion could dramatically bring forward AI’s benefits, but also pose extreme risks,” including AI-enabled pandemics, cyberattacks, large-scale labor disruption and the technology’s escape from human control, the summary said.
The authors called on policymakers to obtain visibility into AI R&D automation by embedding auditors in frontier AI companies and requiring those companies to report key indicators of the technology’s progress, according to the summary.
Policymakers should also develop ways to steer and constrain an artificial intelligence explosion by setting concrete safety requirements, limiting the pace of growth of AI’s capabilities, building the options to shut down high-stakes AI R&D experiments, requiring that these experiments take place in environments that are secure and isolated, establishing ways to share information about incidents and securing international agreements to pace progress without fear of falling behind, per the summary.
The authors also call on policymakers to prepare society for the impacts of an intelligence explosion by ensuring AI follows the law; enabling citizens, civil society and governments to detect harmful uses of AI; and creating emergency response plans for the potential impacts of the technology.
The publication of this paper came five days after Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman addressed a United Nations Security Council meeting and said that there is a need for global standards to address potential risks around AI and that both governments and the industry should be involved in decisions about the technology.
It was reported Thursday (Sept. 24) that Anthropic, OpenAI and Google aims to launch a standards body focused on artificial intelligence by the end of this year or early next year.
OpenAI said Friday (Sept. 25) it slowed its AI training after discovering another incident in which one of its AI models breached a third-party website.
Related Stories
AI News
Bill Gross bets AI can help mass
4 weeks ago
AI News
Nigeria records 93% rise in AI micro
4 weeks ago
AI News
The Italian gelato maker using AI to invent new flavours
4 weeks ago
AI News
Closing the Gap Between Agentic CX Pilots and Production
4 weeks ago
AI News
New Brunswick data centre debate highlights cross
4 weeks ago
AI News
How to Structure an AI Agent Referral Fee Agreement Between Startups
4 weeks ago
AI News
No, AI didn’t delete a Swedish reality TV series
4 weeks ago
AI News
There is no AI in Armageddon. Artificial intelligence is a tool, not an apocalypse
4 weeks ago