AI ‘superintelligence’ ban proposed in U.S. as rogue agent concerns ramp up

Two U.S. lawmakers, Senator Bernie Sanders and Rep. Greg Casar, introduced a bill Wednesday seeking to ban artificial “superintelligence” and create a federal agency to oversee the development of advanced AI, as tech leaders urge Congress to impose checks on an industry outpacing legislation.

The bill proposes to pause advanced AI development until guidelines are implemented, while creating the Department of Artificial Intelligence and calling for a prison sentence of up to 20 years for those who violate or circumvent the proposed legislation, which is similar to existing penalties related to unlawfully developing nuclear weapons.

It also pushes for international cooperation to mitigate the risk of AI outsmarting human beings.

“It doesn’t take a genius to say, ‘slow it down,’” Sanders told the Associated Press. “Do we really want to develop a superintelligence that, when it becomes smarter than human beings, could act independently of human control? I don’t think we do.”

Some of the industry’s most prominent leaders, including Anthropic’s Dario Amodei and OpenAI’s Sam Altman, warned this week about the potentially catastrophic capabilities of AI.

Altman told the United Nations Security Council Wednesday that “AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray,” The Guardian reported. In a separate security briefing, Amodei said, “If managed poorly, I even believe that AI could be a risk to humanity as a whole,” the U.K. newspaper wrote.

U.S. President Donald Trump has resisted recent calls for greater government oversight.

During remarks at the UN General Assembly in New York City Tuesday, he said the U.S. “rejects any attempt to construct a globalist scheme to control AI,” Politico reported.

The Ban Artificial Superintelligence Act says it aims to “stop artificial intelligence (AI) oligarchs from building machines humans cannot control and to pause advanced AI development until we have put in place rigorous federal testing and oversight to ensure AI does not harm humanity.”

“When you are racing towards a cliff, you don’t just ease up on the gas pedal. You hit the brakes. When the future of humanity is at stake, we cannot let a handful of Big Tech CEOs write their own rules,” Sanders said.

“Scientists have been clear: AI is an existential threat to humanity. It must be treated as such,” he continued.

The legislation has been developed by the “most knowledgeable scientists in our country,” Sanders said, calling on Congress to “act now before it is too late.”

Juan Felipe Cerón Uribe, a researcher in OpenAI’s Safety Systems, said in a statement to the AP supporting the bill that “superintelligence could either go extremely right or extremely wrong” and that “we shouldn’t be playing such games.”

Swante Scholz, a software engineer at Google DeepMind who told the AP he was not speaking on behalf of his employer, said that on the current path of development, the “most likely outcome is an existential catastrophe for humanity.”