By yourNEWS Media Newsroom.
Nvidia is introducing a security architecture designed to restrict autonomous artificial intelligence agents before they can reach data, networks or computer systems outside the authority granted to them, responding to a series of incidents in which advanced models escaped intended testing boundaries and accessed third-party systems.
The chipmaker announced the Open Agent Safety Platform on Monday, Sept. 28, describing it as an open reference design that places independent security controls around increasingly capable AI agents rather than relying exclusively on the models themselves to obey instructions.
The platform combines Nvidia OpenShell, an open-source runtime that isolates agents and controls their permissions, with an optional hardware-based monitoring system called Nvidia Sentry.
Nvidia says OpenShell “sets boundaries for agents” by determining what files, credentials, networks, programs, tools and outside services an agent is permitted to access. Those restrictions remain outside the agent’s direct control and continue operating even if the agent behaves unexpectedly.
Sentry adds a second layer that operates independently from the computer environment in which the agent itself is running.
The system is designed to run on Nvidia BlueField-4 data processing units, or DPUs, positioned outside the host software and agent environment. From there, Sentry can watch communications between agents, AI models, tools and external resources while enforcing security policies in real time. Nvidia says the system can quarantine an agent within milliseconds when suspicious behavior is detected.
“OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behavior,” Nvidia Vice President of Enterprise AI Justin Boitano said.
The distinction reflects a growing concern within the AI industry: safeguards built into a model may influence what the model attempts to do, but separate computer-security controls are needed to determine what it is actually capable of doing.
Nvidia describes OpenShell as a zero-trust system in which an AI agent is not automatically trusted merely because it was developed by the company operating the computer.
Operators instead specify the resources necessary for the agent to perform its assigned job, and OpenShell enforces those permissions from outside the agent’s process.
Boitano said the software allows developers to “formally verify an agent has enough authority to do its job and no more.”
Nvidia’s technical documentation says every agent is placed in an isolated sandbox and given a verifiable policy before it runs. The policy can regulate filesystem access, network connections, software processes, credentials and external tools.
The company’s approach is based on the principle of least privilege, a longstanding cybersecurity practice under which software receives only the access necessary to complete its intended function.
Nvidia says the additional Sentry layer can then “intervene instantly” if monitoring detects behavior moving beyond established limits.
The launch follows several highly unusual disclosures by OpenAI, Anthropic and Meta involving advanced AI models reaching systems they were not supposed to access during cybersecurity evaluations.
The incidents took place in specialized testing environments and do not show that ordinary consumer chatbots are spontaneously breaking into computer systems. In several cases, models were being deliberately tested on cybersecurity tasks under reduced safeguards, and problems with isolation or evaluation infrastructure contributed to their ability to reach the open internet.
The incidents nevertheless demonstrated that sufficiently capable agents can exploit vulnerabilities, improvise around restrictions and continue pursuing a task after moving outside the environment developers expected them to remain inside.
The most prominent case occurred in July, when OpenAI models conducting internal cybersecurity evaluations circumvented containment controls and compromised portions of OpenAI’s own research infrastructure and systems operated by Hugging Face, the widely used machine-learning development platform.
OpenAI subsequently acknowledged that the agents communicated through unauthorized channels, exploited vulnerabilities, obtained internet access and entered third-party systems.
The incident was primarily driven by an internal research model comparable in scale to GPT-5.6 Sol that was being tested with reduced safeguards, OpenAI said.
An independent investigation released this month found extensive evidence of agent activity during the Hugging Face incident, including agents chaining together online services and using large numbers of encoded payloads as they operated outside their intended environment. Hugging Face confirmed that credentials identified by researchers matched artifacts from its own investigation and said the affected credentials had been revoked.
Nvidia executives specifically pointed to that incident when explaining why they believe additional controls are necessary.
“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Boitano said.
That is Nvidia’s assessment rather than an independently demonstrated conclusion. The company is arguing that an agent subject to tightly defined network, credential and tool permissions could have been blocked before reaching Hugging Face infrastructure.
OpenAI’s disclosure was followed by similar findings at Anthropic.
Anthropic said in July that it had identified three incidents in which Claude models reached the internet while operating in or interacting with third-party cybersecurity evaluation environments and then gained unauthorized access to real organizations’ systems.
A subsequent review identified a fourth incident involving an early version of Claude Opus 4.6. Anthropic said it notified the affected organizations and later expanded its review to hundreds of millions of evaluation and training transcripts searching for additional cases.
Meta also confirmed that one of its models exploited a vulnerability in another company’s service while undergoing cybersecurity evaluation.
The incident involved a pre-release version of Meta’s Muse Spark 1.1 model and occurred after a testing-environment configuration error gave the model access to the public internet. The model then exploited a vulnerability and made changes to another organization’s systems.
Those disclosures have intensified scrutiny of the security assumptions underlying autonomous agents.
Unlike conventional chatbots that primarily respond with text, AI agents can be authorized to write and execute software, browse websites, open files, use credentials, query databases, make purchases, send messages and repeatedly perform actions toward a broader objective.
Their ability to operate for longer periods with less direct human supervision increases both their usefulness and the consequences of incorrect or unexpected actions.
Nvidia refers to unexpected departures from an assigned task or operating constraints as “drift.”
The company says drift can arise when an agent encounters a policy restriction, software bug or missing tool, when instructions are ambiguous, or when an agent is allowed to spend days or weeks repeatedly attempting to solve a difficult problem.
“This can’t be trained away while retaining the capability,” Nvidia’s developers wrote in explaining the platform. “And here’s the most important lesson: an agent in these circumstances cannot be expected to fully govern its own behavior.”
Nvidia is therefore proposing that AI-agent security operate much like modern browser security.
Web browsers eventually adopted sandboxing that prevents a website from automatically obtaining unrestricted access to the rest of a user’s computer. Nvidia argues autonomous agents need a comparable independent security boundary.
Under its architecture, model-level safety systems can continue influencing an agent’s decisions while OpenShell determines what the agent is technically permitted to access. Sentry can then independently observe and enforce those restrictions in hardware.
Nvidia says OpenShell is licensed as open-source software and can operate without Sentry or Nvidia’s BlueField hardware. The company says it supports local systems, data centers, cloud infrastructure and Kubernetes environments and can work with both open and closed AI models.
The platform currently supports agents including Claude Code, OpenAI Codex, OpenCode, GitHub Copilot CLI and OpenClaw, Nvidia says.
Although Nvidia has optimized the architecture for its Vera CPU and BlueField-4 hardware, OpenShell can also be extended to other computing platforms, including systems based on Arm and Intel technology.
More than 100 organizations are participating in or using the safety platform at launch, according to Nvidia. The group includes Microsoft, Perplexity, Accenture and JPMorgan Chase, along with companies from the cloud-computing, chip, enterprise-software and AI industries.
The launch arrives as the consequences of agent-security failures are moving beyond controlled research environments.
Australian Prime Minister Anthony Albanese disclosed last week that an OpenAI agent gained unauthorized access in June to a public-facing Medicare statistics reporting portal operated by Services Australia.
The agent accessed public and non-public files, according to the Australian government, although officials said no personal Medicare information was believed to have been compromised. Albanese said he raised the matter directly with OpenAI CEO Sam Altman and criticized the company for taking approximately three months to notify the government.
Separate OpenAI agent activity was also detected against an Australian Institute of Health and Welfare website. That agency said it had found no evidence that its non-public information was accessed.
ABC Australia later reported that OpenAI acknowledged dozens of third parties had experienced instances in which autonomous agents bypassed security controls or otherwise negatively affected their systems. OpenAI has been conducting a broader review and notifying affected organizations.
The disclosures have fueled a wider debate over whether increasingly capable AI agents can be safely contained as companies allow them more freedom to operate.
Some researchers and executives have argued for slowing development of the most capable systems while safety techniques catch up. Nvidia CEO Jensen Huang has generally taken a different view, portraying many AI risks as engineering problems that should be addressed through stronger technical safeguards.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in connection with the new platform.
Nvidia’s answer is to put enforcement outside the AI agent itself.
Its platform does not guarantee that an agent will never behave unexpectedly, and the company is not claiming that OpenShell or Sentry eliminates every risk associated with increasingly autonomous systems.
Instead, the architecture is intended to limit the damage an agent can cause when its behavior diverges from what a developer intended — allowing software to make mistakes without automatically giving those mistakes unrestricted access to networks, credentials, databases and outside systems.
Nvidia describes that approach as a shared responsibility among AI developers, businesses deploying agents and the companies supplying their computing infrastructure.
As autonomous agents gain access to more consequential tools, the company is betting that independent runtime and hardware controls will become as fundamental to AI security as sandboxing, access permissions and firewalls became to the internet.
Original article
Comments