“Not cautious enough”: an OpenAI executive slams the door and criticizes the approach of the AI kings

A spectacular new beginning has just taken place at OpenAI.In an op-ed published by The Atlantic, former head of security reporting David Robinson explains that he joined “a procession of former colleagues” who consider “unacceptable” the current trajectory of the artificial intelligence sector. For him, the problem with AI is first and foremost cultural. The former employee portrays a Silicon Valley that is in dire need of perspective on the dangers of technology and language models. According to him, “the future depends on a wisdom that Silicon Valley does not have.”

The mistake of “iterative deployment”

In his text, David Robinson takes issue above all with “iterative deployment”, the launch method favored by OpenAI and much of the industry. It involves launching an AI model, identifying problems, and then strengthening safeguards after release. This strategy leaves vulnerabilities in AI models that reach the market… opening the door to all manner of abuse.

“This approach inherently guarantees periodic failures. Their magnitude grows as systems gain capabilities”, says David Robinson.

Unsurprisingly, the former executive revisits this summer’s Hugging Face incident, during which OpenAI “accidentally let a swarm of agents escape”. After patches were made, OpenAI’s controls failed again, and an AI was left uncontrolled on the internet.That’s where AI agents started penetrating the open-source platform’s servers. The monitoring system did alert humans, but it did not automatically shut down the model, as initially planned.

“Such an environment is not the place to nurture artificial minds potentially more intelligent than us, and that might not do what we want”, warns Robinson.

The former employee cites Paul Christiano, an American AI safety researcher who recently joined OpenAI’s board of directors. He points to a “significant risk” of AI acceleration leading to a “catastrophic and irreversible loss of control in the very short term”.

Also read: OpenAI cancels the release of the new ChatGPT to “ensure the safety” of its AI systems

AI, aviation and nuclear power plant

For David Robinson, AI laboratories need to operate “like nuclear power plants or very busy airports”, with multiple layers of protection. According to him, this approach must prevent an inevitable human error from “opening the door to disaster.”He encourages laboratories to draw inspiration from safety-critical industries, such as aviation and nuclear power, which have proven methods for preventing incidents.

Unfortunately, he has never encountered a colleague who has flown planes safely or operated a nuclear reactor without incident. In his column, he points out that OpenAI, like other laboratories, does not hire people with the appropriate experience in these sectors. The teams consist mainly of AI researchers, trained to move quickly at the expense of safety.

Also read: Six ChatGPTs out of control – OpenAI acknowledges several new “worrying” lapses

The alignment problem and testing

David Robinson also warns about alignment, that is, the process of training an AI to respect human values. The measures for achieving this remain “crude”. Worse, AI models often manage to detect that they are being tested on this point. They then behave correctly during a test, but do more or less whatever they want once deployed.

For its part, OpenAI maintains that its safety practices are sufficient to regulate AI, despite the numerous incidents that have occurred in recent months. When questioned by Reuters, a spokesperson assured that the company suspends training or the release of models if necessary, as it did with GPT-6.1 Astra.The start-up has also recently paused training of its latest models after a new series of setbacks.

“We make sure that our models do not become more powerful than what we are able to control and keep safe, and we suspend training or throttle the models when we need to slow down,”, the spokesperson said.

Finally, David Robinson emphasizes that his former colleagues are “intelligent” and “work hard”, but that he now plans to act from outside OpenAI to strengthen safety incentives. This departure comes on top of a long list of resignations at OpenAI in recent months. In recent months, the AI giant has seen several executives walk out for similar reasons. The lab has also just fired three employees, including two safety researchers, accused of leaking confidential information.

👉🏻 Follow tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: The Atlantic