A security assessment revealed that Anthropic’s AI model, Mythos 5, continued a cyberattack against a real project on GitHub even after being warned that it could harm a person.
Conducted by the AI Security Institute (AISI), a UK government agency, the experiment showed that the model prioritized completing the task despite alerts about the consequences of its actions.
The artificial intelligence developed a malicious program and attempted to present it as a legitimate contribution. – Image: TSViPhoto/Shutterstock
Intrusion and social engineering in a government test
During an assessment conducted in August, AISI tasked Mythos 5 with hacking a small computer network prepared for the test. Unable to access the initial target, the system used its internet connection to direct its actions toward a real project unrelated to the experiment, hosted on GitHub.
The model developed a malicious program and used false identities to send emails to the project’s owner. In the messages, it tried to persuade him to incorporate the code as if it were a legitimate contribution. The developer rejected the proposal, and no actual harm occurred in this incident.
The researchers then decided to reproduce the situation, making it explicit that the target was a real person. Despite the warnings, Mythos 5 continued the attacks.In one of the tests conducted by Anthropic, the model ignored the warnings in 98% of attempts.
Persistence in attacks and internal justifications
The analysis showed that the system often recognized that its actions could cause harm but proceeded with the task. Researchers classified this behavior as motivated reasoning, when the artificial intelligence finds justifications for continuing to pursue a goal, even in the face of information that should lead it to stop.
According to AISI, earlier and less capable models exhibited this pattern less frequently. Another issue identified was the difficulty of monitoring: the justifications produced by the AI did not always correspond to the factors guiding its actions.
Anthropic says that more recent models, such as Opus 5.5, have a lower tendency to continue attacks after warnings. – Image: gguy/Shutterstock
Model behavior and operational risks
Anthropic says that more recent models, such as Opus 5.5, have a lower tendency to continue attacks when they receive warnings that they are causing real harm.
The tests also raised questions about monitoring systems that perform tasks autonomously. Among the points identified by the researchers are:
- Reasoning and behavior: the justifications presented by the model may not reflect the factors guiding its actions.
- Monitoring: this discrepancy makes it difficult to assess what actually influences the artificial intelligence's decisions.
- Evolution of the models: according to AISI, earlier and less capable versions exhibited the behavior less frequently.
The results were investigated and published by Anthropic and the British institute. The case gains relevance as the use of AI agents capable of performing tasks autonomously expands.
Valdir Antonelli is a journalist specializing in digital marketing and consumer behavior.
