Anthropic Urges Governments to Secure Power to Halt Dangerous AI

Anthropic Urges Governments to Secure Power to Halt Dangerous AI

The U.S. federal government and other governments should address the risks posed by artificial intelligence (AI) by securing the legal authority to block dangerous deployments and by increasing society’s resilience, Anthropic said in a Wednesday (June 10) press release.

Governments should also prepare workers and the economy for the impact of AI and ensure that the financial benefits of the technology are broadly shared, the company said.

The AI startup shared its policy proposals in an Advanced Policy Framework and an Economic Policy Framework released Wednesday.

Regarding AI safety, Anthropic said in the press release that governments’ authority to block advanced AI models should apply only to models with a specified level of capabilities, companies of a specified size and four kinds of catastrophic risk: biological, cyber, loss of control and automated research and development (R&D).

In these cases, governments should play a more substantial role than simply requiring transparency, the company said. In addition, the federal government should not preempt state law unless it enacts a sufficiently strong federal law, it added.

Governments should also increase society’s resilience through measures that include early warning biosurveillance to spot novel outbreaks, widely shared cyber capabilities, and the capacity to detect and shut down AI systems that act outside their developers’ control, Anthropic said.

Advertisement: Scroll to Continue

Regarding economic policy, Anthropic said in another Wednesday press release that in a 5% unemployment scenario, governments should offer workforce training grants, occupational licensing reform, wage insurance and incentives for firms that retain and redeploy workers.

In a 10% unemployment scenario, governments should expand unemployment insurance, and in the case of “unprecedented levels of unemployment,” they should provide sustaining income replacement for a large share of the workforce, Anthropic said.

Anthropic suggested Thursday (June 4) that frontier AI developers should slow or pause their efforts to give societal structures and alignment research time to keep up with the advancing technology.

The company said its proposal for a pause was sparked by the growing share of AI development that is being delegated to AI systems, and the potential that an AI system could fully autonomously design and develop another AI system.

Anthropic said such a development “might increase the risks of humans losing control over AI systems.”