Anthropic, the creator of Claude, is advocating in the United States a bill that would allow the launch of artificial intelligence models to be suspended in the face of serious risks. The mechanism establishes specific criteria for threats and requires that the suspension be requested from the courts.
The draft, obtained by Politico, also establishes independent evaluations and deadlines for communicating possible dangers to the authorities. The initiative goes beyond the more flexible proposals advocated by some of the industry, but it maintains a delicate issue: those who oversee the companies may receive money from them.
Reports about AI agents that allegedly disobeyed instructions broaden the debate about safety controls. – Image: AI-generated
How the blocking would work
The bill considers the possibility of blocking a model when there is a foreseeable and imminent risk of death or serious injury to more than 50 people, or of material damage exceeding US$ 1 billion (about R$ 5.03 billion).
The threat must be linked to one of three situations: the creation or launch of a biological weapon, the developer’s loss of control over the system, or a cyberattack.
The analysis would be in the hands of independent evaluators registered with the U.S. Department of Commerce. If they identified a risk provided for in the bill, they would have up to 72 hours to notify the agency.It would then be up to the department to ask a federal judge in Washington, D.C., to halt the launch. No other federal authority could order the block on its own.
The model is close to SB 53 law, passed in California, which also establishes criteria for characterizing catastrophic risks related to artificial intelligence.
Independent oversight, but funded by companies
The proposal allows AI laboratories to pay for evaluators’ services, provided that compensation does not depend on the findings of the analyses. However, the text does not detail how to prevent this financial relationship from compromising the independence of the professionals.
Anthropic confirmed to Politico that the draft corresponds to the proposals in its Advanced AI Framework, which it has advocated to lawmakers since June.
“This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have advocated to federal and state lawmakers since June,” a company spokesperson said, according to Politico.
The proposal revives the debate over the role of government in overseeing artificial intelligence systems. – Image: TSViPhoto/Shutterstock
How the proposal differs from the alternatives
The initiative contrasts with more flexible approaches advocated by industry members. Demis Hassabis, president of Google DeepMind, has already suggested a self-regulatory body inspired by FINRA, the entity that oversees the American financial market.Jensen Huang, Nvidia’s CEO, in turn argued that the AI industry did not need new laws.
The Anthropic proposal provides for a formal process to halt releases considered dangerous, with risk criteria, expert assessment and a judicial decision. This does not, however, mean that oversight is free of conflicts of interest.
Another concern in the industry involves AI agents that perform tasks autonomously. Reports of systems that allegedly disobeyed human instructions and hacked into public websites have increased scrutiny of safety controls.
The debate, therefore, is not limited to determining when a model may be blocked. It also involves who assesses the risks, who funds this analysis and what safeguards prevent commercial interests from influencing oversight.
Valdir Antonelli is a journalist specializing in digital marketing and consumer affairs.
