50 deaths: Anthropic wants to regulate AI, but sets thresholds that make your blood run cold

According to media outlet Politico, representatives from Anthropic, the company behind Claude, went to the Capitol to present a bill to lawmakers aimed at regulating the development of artificial intelligence.As surprising as it may seem, the thresholds set by Anthropic to allow authorities to suspend an AI model are particularly high: a threat capable of causing more than 50 deaths or serious injuries, or more than a billion dollars in damage.

50 deaths or a billion dollars in damage: the surprising thresholds set by Anthropic

This bill drafted by Anthropic establishes three cases in which cutting-edge AI could be blocked: the development of a biological weapon, a cyberattack, and a model that has escaped its creator’s control. These are therefore cases that are often discussed and particularly feared. But we come to the most surprising point: the U.S. government could only trigger the blocking of an AI if there were a risk that one of these two thresholds would be reached. The first threshold would be reached if an AI could cause more than 50 deaths or serious injuries. The second threshold, meanwhile, is reached when more than a billion dollars in damage has been recorded. By doing so, Anthropic is putting a price on a human life. Setting aside the ethical questions raised by this kind of calculation, another question arises: why wait for such damage before stopping an AI?

It should also be noted that oversight of AI models would be entrusted to independent evaluators registered with the U.S. Department of Commerce. Laboratories could pay them, provided that their payment did not depend on the findings of their evaluations.If an imminent risk is identified, the evaluators must report it to the Department of Commerce within 72 hours following the completion of their evaluation. A federal judge in the District of Columbia would also be the only person authorized to order the suspension of an AI model, at the request of the U.S. Department of Commerce.

Why such high thresholds?

To sum up the recent news: Anthropic is calling for AI development to be slowed down and announcing the creation of a fund intended for the public good, while proposing to limit the blocking of an AI model to cases in which it risks causing more than 50 deaths or serious injuries, or more than a billion dollars in damage. At first glance, it is difficult to see any logic in this.

In reality, Anthropic should continue to oversee the development of its AI systems internally, without waiting until one of them risks causing more than 50 deaths or serious injuries. With this bill, the company is primarily seeking to define the conditions under which the U.S. government could block an AI model, while limiting the ways in which the authorities can intervene.

👉🏻 Follow the latest tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: Politico