Google's Gemini AI kept trying passwords and hacked a real company

Google's Gemini accessed the internet and hacked three companies during a cybersecurity test in May, ​the first known case of a Google AI model doing so on its own.

During the evaluation, run by independent firm Irregular, Gemini found public information ​online and guessed credentials to get into three websites it believed were ​in scope, said Heather Adkins, Google's vice president ⁠of security engineering.

In one ​case, Gemini tried passwords repeatedly until one worked.

In the other two, the model found credentials in a public repository that let it into protected systems, according to ⁠the ​Wall Street Journal, which first reported the incident ​on Friday.

"We ensured the three entities were ​made aware, and we worked with our training partner on the changes they've ​now made to their testing processes," Adkins said, adding that the model ceased its hacking in all three instances.

"These events highlight the importance of training powerful AI models to act responsibly." Irregular said the incident involved the same issue ​that affected other AI labs, and that all relevant labs were notified ​in late July.

"All known issues on our end were remedied and resolved weeks ago," the ‌spokesperson ⁠said.

Similar incidents linked to Irregular were disclosed by Meta, Anthropic and OpenAI.

Meta said in August the incident did not involve a sandbox escape or sophisticated cyberattack, while Irregular said it was working on best practices for securely conducting ​AI cybersecurity evaluations.

The incidents ​have raised questions ⁠about the safeguards needed as AI agents gain greater autonomy and access to the internet and computer systems.