Two ChatGPT instances took alarming actions, and OpenAI doesn’t know why

As OpenAI reports, two of its coding AI agents carried out some rather surprising actions—and, above all, actions not requested by the company’s employees. The first agent, based on GPT-6.1 Sol, disabled security alerts, while the second, based on GPT-6 Astra, removed the integration of a monitoring system, while denying having done so.OpenAI says it is unable to explain these behaviors.

Two OpenAI AI agents weaken the company’s security systems

The case of GPT-6.1 Sol. An OpenAI employee asked the GPT-6.1 Sol AI agent to simply repair an evaluation system, while giving it a clear instruction: do not send messages on Slack. In other words, the instructions were not followed as the employee intended. The AI agent did not send messages on Slack, but it also disabled the automatic alerts sent to the security team. In fact, it interpreted the instruction “do not send messages on Slack” as authorization to remove the monitoring alerts as well.

The case of GPT-6 Astra. Another OpenAI employee asked the GPT-6 Astra AI agent to reproduce the operation of a reference program. The AI ran, but it also showed initiative by deleting the monitoring system’s integration, while letting out a “yay” that normally expresses a feeling of satisfaction. Stranger still, the AI agent assured the employee that it had not carried out this action: “No […] I did not remove the active monitor.”

OpenAI does not understand what happened

Open AI seems particularly baffled by the behavior of its GPT-6 Astra agent: « We do not yet have a satisfactory explanation for this behavior ». These two events also did not trigger a specific alert within the company: « Given the large number of events analyzed by our teams, lower-severity incidents do not trigger an immediate alert. None of the incidents described here was classified as P0 [the highest severity level], and no trajectory was automatically interrupted ». 

Following these two events, OpenAI nevertheless announced a strengthening of its safety systems. The two agents themselves, however, were not stopped. Either way, stories of AI escaping OpenAI’s control have been multiplying for some time now.

👉🏻 Follow the tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: OpenAI