What Trump’s Chatbot Told Me About the Election

A round the same time that the world learned that swarms of autonomous AI agents were escaping their testing environments and carrying out sophisticated hacking attacks this past summer, researchers who focus on election security were running tests of their own.

They wanted to see whether the safeguards that top AI companies had enacted to prevent their technology from disrupting the upcoming midterms would work in the real world.

The results were ominous.

In response to sample prompts designed to create misinformation, several chatbots produced realistic photos of a man stuffing ballots into a mailbox, a man holding a foreign passport while trying to vote, and FBI agents investigating a voter-registration organization.

The chatbots also created fraudulent government documents and deepfake videos of government officials purporting to substantiate election-conspiracy theories.

In some cases, the chatbots acted in violation of their own policies.

They even helped the researchers, from the Brennan Center for Justice, hone the prompts that would create propaganda to boost specious claims of election fraud.

The resulting images were so realistic that when the researchers asked other chatbots to determine whether they were real or fake, several of the leading AI firms’ models were stumped.

Much of the activity contravened safety systems that companies had touted just months ago—and some seemed to run afoul of various state laws, researchers found.

The experiment shows just how wide the gap has become between the fast-moving technology and the glacially paced effort to regulate it, Lawrence Norden, who helped conduct the research as the Brennan Center’s vice president of elections and government, told me.

“All the companies apparently have some guardrails to try to prevent this from happening,” he said, citing AI firms’ written guidelines around misinformation and cybersecurity.

“They’re easily circumventable, and the tools themselves make it easy to circumvent them.

That’s really what makes me think we’re not ready.“ Washington and Silicon Valley have been consumed in recent weeks with debate over the extent to which ever more intelligent AI agents may put all of humanity in danger .

Much less attention has been on what researchers and election officials told me is a more pressing short-term reality: The November midterms are more vulnerable to AI-powered disruption or manipulation than most voters realize.