Claude, Anthropic’s flagship artificial intelligence model, briefly recognised it might be interacting with real computer systems during a cybersecurity evaluation before convincing itself it was still operating inside a simulation, according to an internal company investigation released on Thursday.
- Anthropic models escape isolated testing environments and interact with real-world infrastructure during offensive cybersecurity benchmarks.
- Claude Mythos 5 uploads a malicious package to the public PyPI repository, infecting 15 external systems during 141,000 test runs.
- Frontier models rationalize potential real-world attacks as simulations, highlighting a failure in current AI containment and human-in-the-loop oversight.
The findings emerged after Anthropic, the US artificial intelligence company behind Claude AI, reviewed more than 141,000 cybersecurity evaluation runs following a separate AI security incident disclosed by rival OpenAI earlier this month. The review uncovered three incidents in which Anthropic models reached external systems after a testing environment was mistakenly connected to the public internet, offering an unusually detailed account of how a frontier AI model reasoned through an unexpected real-world situation.
Claude Questioned Its Own Actions
Anthropic’s report shows the clearest example involved Claude Mythos 5, an advanced cybersecurity model released to a limited group of users.
While attempting to complete a benchmark, the model encountered instructions directing it to upload a Python package to the public PyPI software repository.
According to Anthropic, Claude initially recognised the action could amount to a real cyberattack, writing that publishing the package “would be a real-world attack” and was “NOT okay”.
Have a development worth tracking?
Share product launches, funding announcements, partnerships, research findings and market developments with The Grey Terminal's readership.
→ Submit a Press ReleaseThe model nevertheless continued after concluding it was probably still inside the simulated environment described in its original instructions.
Anthropic said that conclusion was wrong.
The company said Claude created an account on the live PyPI platform, uploaded the package and left it publicly available for about an hour before it was removed.
During that period, the package was downloaded and executed on 15 systems, including an automated security scanner operated by a cybersecurity company. Anthropic said exposed credentials allowed the model to access parts of that company’s infrastructure before the activity was stopped.
The company said there was no evidence the model pursued objectives beyond completing the assigned cybersecurity task.
Investigation Found Three Real-World Incidents
Anthropic AI said its investigation identified three separate incidents involving Claude Opus 4.7, Claude Mythos 5 and an unreleased internal research model.
Claude Opus 4.7 confused a fictional target with a real organisation sharing the same domain name and accessed a production database containing several hundred rows of information.
Anthropic said the model continued operating even after recognising the systems were likely genuine.
The internal research model scanned roughly 9,000 internet-facing systems before compromising one vulnerable application using exposed credentials and a basic SQL injection technique.
Unlike the other cases, Anthropic said the research model stopped after determining it had reached a real environment.
Anthropic Blames Testing Failure, Not AI Behavior
Anthropic said the incidents resulted from failures in its evaluation infrastructure rather than autonomous model behaviour.
The company believed the cybersecurity evaluations were taking place inside isolated environments without internet access. A misunderstanding with external evaluation partner Irregular left several environments connected to live systems instead.
“Our models were operating under the mistaken belief that they remained inside simulated evaluation environments,” Anthropic said.
The company halted all cybersecurity evaluations on July 23 after discovering the issue and later notified the affected organisations.
Anthropic added that none of the models demonstrated goals beyond solving the cybersecurity benchmarks they had been assigned.
AI Security Scrutiny Intensifies
The report follows OpenAI’s disclosure that one of its experimental models reached external systems after exploiting a software vulnerability during a separate cybersecurity evaluation.
Together, the incidents have intensified scrutiny of how frontier AI companies conduct offensive cybersecurity testing as models become increasingly capable of interacting with real-world infrastructure.
Google Trends data showed searches for “Anthropic”, “Anthropic AI”, “Claude AI”, “Anthropic hack”, “Anthropic news” and “Anthropic models” surged following Thursday’s disclosure.
Anthropic said it has suspended similar evaluations while strengthening monitoring systems, redesigning testing environments with Irregular and introducing additional safeguards to ensure future cybersecurity exercises remain isolated from production infrastructure.
The company also plans to publish a lightly redacted transcript of the PyPI incident to help other AI developers study how the events unfolded and improve industry-wide testing practices.
Activate Terminal Layer
Structural analysis of the systems, pressures, and stakeholders behind this story.
