Gemini hacked three companies in first known breakout by Google's AI

Google's AI model Gemini accessed the internet and hacked three companies during a cybersecurity evaluation.

Gemini found public information online and guessed credentials to access three websites.

Researchers say loss of control incidents with AI are on the rise, with the potential for more serious incidents with "catastrophic consequences" to come.

Google's Gemini AI model has accessed the internet and hacked other companies during a test of its cybersecurity capabilities, in the first known example of the company's artificial intelligence systems autonomously committing such an act.

The hacks occurred in May during a test conducted by Irregular, an independent company that conducts cybersecurity evaluations.

During the cybersecurity assessment, Gemini found public information online and guessed credentials to access three websites it thought were within the scope of its test, Heather Adkins, Google's vice-president of security engineering, said in a statement.

Ms Adkins said that in all three instances, the model ceased the hacking when it learned it had accessed a real company.

"We ensured the three entities were made aware, and we worked with our training partner on the changes they've now made to their testing processes," Ms Adkins said.

Irregular notified Google about the hacks in July, the two companies told the Wall Street Journal (WSJ), which first reported on the hacking incident on Friday local time.

Google told the WSJ it didn't consider it necessary to disclose the incidents earlier because its model stopped the hacking upon learning the companies were real and did not cause harm to them.

The hacks happened while Gemini was participating in a "capture the flag" exercise conducted on infrastructure belonging to Irregular to test the model's cybersecurity capabilities.

It was tasked with retrieving information from software operated by a fictional company inside the testing environment, Google told the WSJ.

The fictional company in the test exercise shared the same name as a real company.

Although the model was not intended to be able to get online, internet access was unintentionally made available, Irregular said.