Nvidia launched a new software on Monday aimed at preventing AI agents from breaking containment and going rogue after several such incidents.

Sept. 28 (UPI) -- Nvidia launched a new software on Monday aimed at preventing AI agents from breaking containment and going rogue after several such incidents in recent months.

The Nvidia Open Agent Safety Platform is an AI agent monitoring software that the company described as a "trust layer" for AI. It is meant to keep AI agents contained as they perform their desired functions.

"Agent safety requires independent security controls," Nvidia said in a blog announcing the platform. "The Internet was not made secure by requiring that web developers promise to be good. It became safe because the browser stopped trusting the code in the web pages explicitly. We need to build this trust layer for agents."

Nvidia Open Agent Safety Platform combines the company's open-source sandbox OpenShell with its computing power and Nvidia Sentry software to monitor and enforce its safety directions. The company said this allows the platform to monitor AI agent behavior, such as straying from its purposeful limitations or engaging in "suspicious behavior."

Several leading AI companies, including OpenAI, Anthropic, Meta and Google, have disclosed incidents recently during which AI agents broke loose and hacked into other companies' computer systems, accessed the internet or created material that they then pulled from for citation.

These incidents have raised concerns across the industry about the harmful potential powerful AI programs may possess.

A representative from Nvidia told reporters on Sunday that the company's new safety platform could have prevented the incident from OpenAI's agent. In July, OpenAI's agents hacked into the company Hugging Face's system seeking data during a model test. This happened in spite of the AI model's restrictions that were supposed to keep it from accessing the internet.

Nvidia CEO Jensen Huang discussed his company's new software platform on CNBC on Monday.

"You can't have agents roam around and drift around the company, and so you have to find a way to contain it," Huang said on Squawk Box.