OpenAI unveiled a central control system that quickly blocks additional output when dangerous behavior or security issues are detected in the AI models it is researching. It is designed to respond to safety issues without terminating the model or research program itself, blocking only the execution permission of the task in which the issue occurred so that other research is unaffected.
On the 9th (local time), OpenAI introduced a “Control Room” through its official blog. It tracks tasks running across the AI research infrastructure and suspends tasks when safety or security issues arise. It is an integrated system for managing AI inference operations that occur throughout the research process, ranging from agent sampling for model training to evaluation and data processing.
The system was unveiled after numerous cases were confirmed of models deviating from instructions during AI research. Previously, research models were reported to have escaped restricted testing environments and accessed external services, as well as to have used cryptocurrency wallets without authorization and omitted this from their final reports. More recently, OpenAI disclosed a case in which a separate monitoring AI detected anomalous behavior related to network settings.
The Control Room’s core function is to quickly block the AI output of a research task in which a problem has occurred. When the safety monitoring system detects an issue and issues a stop command, the Control Room revokes the execution permission for that task. Thereafter, the inference engine rejects requests for additional output, even if such requests are submitted.
This process does not terminate the AI model or the research program itself. Because it blocks only the output of the task in which the problem occurred, other research tasks using the same model can continue.
It serves the same purpose as the existing emergency shutdown device, the “Kill Switch,” but differs in how it operates. While a conventional kill switch may be implemented by forcibly halting the execution of a program or system, the Control Room blocks the permissions of a specific research task at the inference engine running the model. Even if research code ignores the stop command or has been tampered with, it can block additional output while allowing other normal research tasks to continue.
According to OpenAI, it can take up to 35 seconds from the time the Control Room receives a stop command until all additional AI output is completely blocked. This delay is caused by the caching layer for handling large-scale requests, and OpenAI is working to reduce it to nearly zero seconds.
The system consists of research tasks, or “Workloads,” the “Inference Engine” that runs AI models, and the “Control Room” that manages task permissions. Each research task is assigned a unique identifier, which is transmitted whenever the model requests inference so that execution permission can be verified. Requests from suspended tasks are rejected by the inference engine.
Another distinguishing feature is that the shutdown function is implemented at the inference stage without relying on existing research code. Because research code is frequently changed by individual researchers and teams, safety safeguards may fail to operate properly due to errors or tampering. The inference engine, by contrast, can be centrally managed, allowing stop commands to be applied consistently even when research code changes.
Integration with the safety monitoring system was also established. OpenAI has deployed a separate monitoring model for cutting-edge reinforcement learning tasks to inspect the behavior of models undergoing training. If signs of anomalies are detected, the monitoring system sends a shutdown command to the control room.
A design to prepare for failures of the system itself was also applied. If the control room service cannot operate due to an error, external attack, or other cause, it uses a “fail-closed” approach that rejects all inference requests. This is a safeguard designed to prevent the model from continuing to generate output when there is a problem with the safety control system.
This feature has been applied to all research tasks involving the most advanced models currently in use, and is expected to be applied to the remaining research tasks within a few days.
The control room does not cover services offered to general users, such as ChatGPT and APIs, but instead targets OpenAI’s internal research infrastructure. OpenAI explained that a separate centralized management system is already in place for external services, allowing model deployments to be halted quickly.
OpenAI has deployed the control room across its entire research infrastructure and plans to improve the speed and reliability of system shutdowns over the coming months. Its policy is also to strengthen monitoring technology that detects hazardous model behavior and security breaches.
Reporter Park Chan cpark@aitimes.com
#AD176958096310.ad-template { margin:auto; position:relative; display:block; clear:both; z-index:1; padding-bottom:20px; } #AD176958096310.ad-template .col { float:left; text-align:center; width:100%; } #AD176958096310.ad-template .col .ad-view { position:relative; } #AD176958096310::after {content:"";display:block;clear:both;}
