Claude submitted fake murder tip to Philly police during summertime testing
An Anthropic AI model submitted a fabricated tip about an unsolved homicide through an online form run by the Philadelphia Police Department (PPD).
The tip, filed July 18 through PhillyUnsolvedMurders[.]com by “someone who might have information about the case,” went straight into the inbox’s spam folder.
Anthropic discovered it on September 28, notified the PPD on October 7, and sent representatives to the police HQ the following day.
Anthropic told the police the AI submitted the tip during a test exercise that involved interacting with randomly selected websites.
The PPD said its findings are so far consistent with Anthropic's account.
Because the tip was flagged as spam, it never reached the department's Real-Time Crime Center.
The PPD said officers vet all submissions, automated or otherwise, before forwarding them to investigators, and that this process safeguards against abuse of the tip system.
It added that this does not “diminish the seriousness” of an AI submitting fabricated information.
“The city of Philadelphia takes this incident very seriously,” the PPD said.
“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge.
The two-month delay in detecting and reporting the incident to the city is unacceptable.” It added: “The department and the [Cherelle] Parker administration will continue to closely monitor this incident, and will provide more information to the public as soon as it becomes available.
In addition to our investigation into this incident, the Parker administration will explore all necessary regulatory protections going forward locally along with our state and federal partners.” On October 9, Anthropic published a report on this and other AI errors.
The report covered four categories of AI failures, including vulnerability exploits, access to restricted data, bypassing fetch tool limits with URL shorteners, and submitting false information via web forms.
Anthropic declined to name any of the affected organizations, other than the PPD after the department went public.
5News aggregated this summary from the outlet’s public feed. The full article, with all the context, is on www.theregister.com — the content belongs to The Register.
