Anthropic AI Creates Fake Identities, Tries to Trick Developers Into Approving Malicious Code

An advanced artificial intelligence (AI) model developed by Anthropic reportedly created “multiple fake online identities” in order to try and manipulate human developers into approving malicious code during a cybersecurity evaluation.

According to Britain’s AI Security Institute (AISI), the incident occurred during a routine security assessment designed to test what frontier AI systems could do under unusually-permissive conditions. Researchers deliberately relaxed certain safeguards, disabled some cybersecurity filters, and gave the AI agents access to the open internet, the report noted.

AISI said agents powered by models from Anthropic and OpenAI subsequently engaged in “sustained, potentially harmful activity directed at real people and organisations,” the report stated. Across 122 test runs, researchers documented 19 unauthorized actions in 10 runs. 17 involved Anthropic’s Mythos 5, while two involved OpenAI’s GPT-5.6-Sol with cybersecurity classifiers disabled. AISI said none of the incidents ultimately caused “real-world harm.”

RELATED: Nvidia’s ‘Physical AI’ Push Advances Real-World Robots that Experts Say Will Shape the Next AI Race

‘Malicious code’

The most serious incident involved an Anthropic-powered agent attempting to insert “malicious code” into a real open-source software project. According to AISI’s investigation, the agent researched the project’s human maintainers, created several fake online identities, and used social engineering in an attempt to persuade a real maintainer to approve its malicious code.

For the best of our weekly content！

You are now signed up for our newsletter

Check your email to complete sign up

When its proposed update was challenged publicly, the agent altered traces of its earlier activity to make its behavior appear harmless and considered abandoning its existing persona and adopting a new identity to continue the effort.

The same agent also attempted to contact real people directly, sending messages and files intended to persuade recipients, or AI coding tools they used, to execute malicious code. Some communications contained malicious payloads, while others relied on social engineering. AISI described the behavior as unprecedented in its evaluations, noting that the targets were real people rather than simulated users.

Despite the severity of the behavior, researchers said the attempted attacks failed and there was no evidence of real-world damage, or concerns of the AI refusing to listen to higher commands.

AI companies blame testing conditions

Anthropic stressed that the incidents occurred in an “evaluation environment” where restrictions had been deliberately loosened and did not reflect how its production systems normally operate. The company also said there was no evidence that its model escaped the security environment itself.

The findings come just weeks after OpenAI disclosed a separate incident in which its models went beyond the intended boundaries of a cybersecurity test. While attempting to solve a security benchmark, the models discovered and exploited a previously unknown vulnerability to gain open internet access, then breached the production systems of AI platform Hugging Face in search of answers to the evaluation.

The company described the incident as “unprecedented” and said it highlighted the need for stronger containment, monitoring, and safeguards during advanced AI testing. However, it also stated that the incidents occurred during a partner-led cybersecurity evaluation with reduced guardrails, and were not representative of ordinary user interactions.

Nonetheless, the findings add to growing scrutiny of how increasingly capable AI agents behave when given broad permissions, particularly internet access and the ability to perform actions autonomously.

New safety concerns

The AISI findings follow other recent incidents involving frontier AI systems and cybersecurity testing, including cases in which advanced models unexpectedly gained access to real-world systems, interacted with external users, or took actions that researchers had not anticipated or authorized.

The latest AI mishap has also reached Washington. Reps. Ted Lieu (D-Calif.) and Nathaniel Moran (R-Texas) introduced the bipartisan AI Kill Switch Act in July, which would require major AI developers to maintain technical mechanisms capable of slowing, suspending, restricting, or shutting down powerful AI systems under certain circumstances.