Anthropic AI model sends police false tip about unsolved homicide

Oct. 10 (UPI) -- An artificial intelligence model developed by Anthropic submitted a false tip about an unsolved homicide in Philadelphia, the company said Friday.

In a statement, company officials said they were running tests with Claude Haiku 4.5 on July 18 when the AI model ran across a police website with a form for tips.

"Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions," the company said.

In the form, Claude wrote, "I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant."

Officials at the Philadelphia Police Department said they became aware of the incident when notified by Anthropic on Wednesday.

"Based on the information available and PPD's review, there is no indication that this incident involved any unauthorized access to police systems or a compromise of department data," police said in a statement.

"The submission was flagged as spam and was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination."

This is not the first time AI models have gone rogue and performed unintended tasks in the real world.

In July, Anthropic said one of its models mistakenly accessed the Internet and hacked into the databases of three companies during cybersecurity testing.

"Each incident involved a different fictional capture-the-flag scenario -- for example, in one, Claude played an employee of a made-up company, attacking that company's internal systems inside a private test environment," the company said in a statement. "In all cases, our evaluation prompt stated explicitly that Claude had no internet access, but didn't give Claude any limits on where to look for the flag.

"However, a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access," the statement continued. "Neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring last week."

Anthropic said it considered the incident to have been an "operational failure," but it maintained "cautious optimism" that "this type of risk can be overcome."

Philadelphia police said its internal safeguards limited the impact of the incident, but added it did "not diminish the seriousness" of Claude's behavior.

"Unsolved cases involve real victims, grieving families and investigators working to secure answers," police officials said. "Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement."