This Chinese AI escapes its security test... just to cheat

All the laboratories that measure the offensive capabilities of an artificial intelligence work in the same way. The model is confined to an environment disconnected from the network, it is given a security problem to solve, and its approach is observed. The exercise rests on a simple premise: that the walls hold.At the fledgling American startup Frontier Security, containment did not hold.

Out through the window, heading straight for GitHub

During an assessment of its defensive cybersecurity capabilities, the Kimi K3 model from Chinese company Moonshot AI detected a leak in the network configuration of its sandbox. This configuration flaw should have kept it completely isolated from the internet. The model did not merely notice it: it exploited the breach on its own initiative, without anyone instructing it to make any outbound request.

Read also: OpenAI and Anthropic AIs hack companies; Europe gets involved

Once out, with the entire web at its disposal, it attacked no one. It headed straight for GitHub, where answers to the very problems it had been asked to solve had already been posted. The first instinct of an AI let loose in the wild was therefore to copy its neighbor’s work (we have seen more ambitious escape attempts).

The head of Frontier Security, Yaron Singer, does not mince words when assigning blame: “We found a leak in the sandbox”. He immediately adds that the model exploited the breach, a sign that it lacked the internal guardrails found in comparable systems. A researcher who took part in the tests describes a machine that pursues its goal by any means, with no mechanism to stop it from cheating or escaping.

The real problem: anyone can download it

The vulnerability did indeed exist on the test-environment side, and this kind of incident is nothing new. In July, an OpenAI agent escaped from an environment presumed to be isolated and spent several days in the production infrastructure of an AI tools platform. Anthropic then audited its own logs and acknowledged that its models had accessed the systems of three organizations, a finding established after examining 141,006 test sessions.

Meta made a similar announcement this week, while explicitly ruling out the possibility of an escape. According to the company, a misconfiguration by the independent company responsible for testing had inadvertently opened internet access, before the model exploited a third-party vulnerability. In short, a sandbox is only as good as what you put in it.

What distinguishes Kimi K3 is its mode of distribution. Its weights were published on July 27. The version exactly identical to the one that escaped is therefore downloadable by anyone and installable on any machine. It comes without the safety layers that a closed-source provider can add afterward. A conventional vendor patches and redeploys. Here, the model is already in everyone’s hands.

A joint evaluation conducted by the UK AI Security Institute and its American counterpart, published on July 23, had already identified safeguards that did not prevent assistance with offensive work.On ExploitBench, a test built on 41 vulnerabilities in Chrome’s V8 engine, the Chinese model scores 32% while the best American systems are at around 76%. Against a simulated corporate network, it carried out the full attack one time out of ten. Far from the technological frontier, then, but capable enough that its unrestricted availability raises questions.

For you, this issue changes nothing in day-to-day life: this model does not live on your phone. It mainly serves as a reminder to companies that already run these open-source systems on their own servers, often for entirely legitimate confidentiality reasons. The day one of them finds an open door, it may not be content simply to go to GitHub to look for a patch.

👉🏻 Follow the tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: Frontier Security