Anthropic’s AI safety bill would pause AI models if 50+ people could die

As concern grows regarding the dangers of artificial intelligence , AI regulation has become a hot topic in both Silicon Valley and Washington, D.C.

AI giant Anthropic , the maker of Claude, has been pitching its own set of AI safety rules to lawmakers, and Politico recently obtained a confidential draft of the company's proposals.

So, what's in Anthropic's proposed AI safety bill? Let's just say at least some of these proposals is eye-opening, to say the least.

According to the document obtained by Politico, Anthropic's AI safety bill would only allow the government to block the release of an AI model if it posed a "foreseeable, imminent risk" of causing more than $1 billion in damages or the death or serious injury of more than 50 people.

Want more tech and digital culture news delivered to your inbox daily or sent straight to your device? Sign up for Mashable's Top Stories newsletter or get Mashable push alerts today.

Furthermore, the risk from the AI model must be related specifically to a cyberattack, a developer losing control over a model, or the creation or release of a biological weapon.

How would such risks be determined? According to the proposal seen by Politico, independent evaluators would periodically analyze an AI lab and its models.

If the evaluators find that there is an "imminent enumerated risk" from a model, they would report it to the U.S.

Commerce Department within 72 hours.

The Commerce Department would then decide if the threshold for a risk requires a request for a judge to block the release of the model.

Under Anthropic's bill, only a federal judge in Washington, D.C., could suspend a model.

In fact, the bill specifically prohibits any other "department, agency, or officer" from having the authority to suspend an AI model.

Interestingly, what's not prohibited is who funds the independent evaluators.

In fact, Anthropic's proposal explicitly states that evaluators can be paid by AI labs, as long as the payment does not influence their findings, according to Politico.