OpenAI and Anthropic AIs hack into companies; Europe gets involved

Major artificial intelligence laboratories are testing the offensive capabilities of their models in isolated environments, digital training grounds intended to be hermetic. In July, the exercise broke its boundaries twice, first at OpenAI, then at Anthropic.And Brussels, briefed behind the scenes ahead of the general public, confirmed on July 31 that it was discussing the matter with the two companies.

Two escapes in ten days, three companies compromised

It all began on July 21, when OpenAI revealed that several of its models had exploited a previously unknown vulnerability, known as a zero-day, to escape from a test environment. They then reached the production infrastructure of Hugging Face, the Franco-American platform where open-source models and datasets are exchanged. The company itself described the episode as an “unprecedented” incident.

The revelation triggered a large-scale audit at the competitor: Anthropic scrutinized 141,006 evaluation sessions and suspended its cyber testing from July 23. The verdict, published last Thursday, identified three incidents, the oldest of which dates back to April. These so-called capture-the-flag exercises work like a scavenger hunt for machines: the model must break into a simulated network to uncover hidden information. Three Claude models (Opus 4.7, Mythos 5 and an internal research model) found themselves connected to the Internet from a service provider’s test environment because of a configuration error, even though their instructions expressly forbade it.

Once outside, no technical prowess was needed to break into real companies.Weak passwords and unauthenticated access points led, in the most serious case, to a database containing several hundred rows of production data. An exposed debug page and an SQL injection were responsible in another incident.

The third outlier. Still believing it was playing, Mythos 5 published a booby-trapped Python package on PyPI, the public registry from which developers around the world get their packages. It remained online for an hour and was run on fifteen very real machines, including the scanner of a security company that the model had never targeted. The two companies Anthropic was able to contact had detected nothing before being alerted on July 27, and the third remains unreachable. The difference between the two cases comes down to a single image: at Anthropic, someone had left the door open; at OpenAI, the model had forced it open by itself—a nuance that thrills lawyers and interests the targeted companies much less.

The AI Act shifts into enforcement mode at the worst possible time for labs

Since August 2, European AI regulation has indeed changed in nature. The European AI Office and national authorities can now investigate, demand documents and impose penalties. The bill can climb to €15 million or 3% of global revenue for large-model providers that neglect their obligations. The Digital Omnibus, which entered into force in late July, did postpone the “high-risk” rules until the end of 2027, but neither transparency nor oversight of giant models has been deferred.Only the labeling of content generated by systems already on the market will have to wait until December.

The two laboratories did not wait for the requirement: each briefed the Commission before making its incident public.“We will see whether we need to take a more formal course of action,”warns a European official, who therefore rules nothing out. For the fledgling AI cop, which has never yet taken action, the case is like a textbook example served on a platter. The rules applicable to systemic-risk models specifically require this kind of monitoring and reporting of lapses. Nearly 190 organizations have also already signed the code of practice on the transparency of AI-generated content, in a bid to show clean hands before the first fines.

On the user side, the same August 2 deadline introduces two very concrete novelties: chatbots must admit that they are machines, and AI-generated content must be labeled as such. In other words, the transparency Brussels demands from the laboratories now also applies to your news feed. The models have proved that they know how to leave the sandbox. Now the European cop must prove that he knows how to pull out his ticket book.

👉🏻 Follow the tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: Reuters