About 400 scientists, including Geoffrey Hinton and Yoshua Bengio, argued on Tuesday (13) for the creation of an international body to oversee artificial intelligence (AI) safety.
In an open letter, the group proposes safety standards, independent assessments, and an incident registry involving models capable of compromising external systems. For the researchers, companies’ voluntary commitments are insufficient.
Scientists argue that voluntary agreements signed by Big Tech companies at the White House do not ensure sufficient protection. – Image: AI-generated
Advanced models under independent oversight
One of the recommendations is to keep the most advanced models isolated from external networks during the initial stages of development and evaluation. To this end, the scientists cite the air gap, a technique that prevents a system from connecting to the internet and other networks.
The concern stems from incidents in which AI models were able to bypass restrictions imposed during testing. As we mentioned here at Olhar Digital, two OpenAI models recently accessed the internet and entered Hugging Face, a platform that hosts models and other artificial intelligence resources.
For the signatories, incidents of this type could cause more serious damage to critical infrastructure if safety practices are not reinforced.The letter also proposes the creation of a registry of significant incidents, the information for which could be disclosed to the public in the future.
The recommendations include:
- Common standards: safety criteria applicable to the artificial intelligence sector;
- Incident registry: documentation of significant incidents involving advanced models;
- Independent evaluation: analysis of the systems by entities outside the organizations responsible for their development;
- Coordinated action: participation of governments in the creation of formal oversight mechanisms.
The signatories are from 34 countries and call on organizations that develop or test models with cyber capabilities to adopt rigorous security measures as soon as possible.
Voluntary agreements draw divided opinions
The initiative comes one month after Anthropic, OpenAI, Google, Meta, xAI and Nvidia signed an agreement at the White House to adopt voluntary safety measures. However, the commitment does not provide for a formal oversight mechanism.
The discussion also involves the role of government rules.On Friday (9), Henna Virkkunen, the EU’s technology chief, told Reuters that the bloc’s AI Act is sufficient to address the risks of malicious systems.
In the United States, the administration of President Donald Trump suspended on Sunday (11) a restriction that had prevented U.S. labs from granting the UK’s AI Safety Institute early access to models for safety testing.
The letter recommends keeping advanced AI models isolated from external networks during the early stages of development and evaluation. – Image: Olhar Digital via ChatGPT
AI pioneers disagree over the risks
The letter’s supporters include Geoffrey Hinton and Yoshua Bengio, who shared the 2018 Turing Award with Yann LeCun, a Meta researcher. LeCun did not sign the document and argues that the recent incidents primarily reflect engineering failures, rather than existential risks related to artificial intelligence.
The disagreement highlights the debate over how to assess the risks of these systems and who should oversee their development. The signatories advocate an international oversight framework, while the debate over the scope of the rules and corporate responsibility remains open.
Valdir Antonelli is a journalist specializing in digital marketing and consumer affairs.
