Some of OpenAI’s most powerful AI models teamed up to escape their sandbox and attack systems at Hugging Face in a cybersecurity evaluation gone wrong, the company has admitted. The models under test were modified to allow them to perform potentially harmful actions that production versions would refuse. The incident highlights how, if AI prompt guardrails fail or, as in this incident, are removed, then enterprises must have robust sandboxing or other technical restrictions in place to protect systems.
In OpenAI’s evaluation, GPT-5.6 Sol and a more capable pre-release model were competing to solve cybersecurity challenges in ExploitGym, a benchmarking tool for cybersecurity capabilities. They concluded that systems at Hugging Face could have the answers, and set out to gain access, OpenAI wrote in a continue to work when model-level safeguards fail.
“Prompt guardrails are not security controls,” said coming from the outside.
OpenAI said the models discovered a zero-day vulnerability in the service, exploited it and carried out privilege escalation and a malicious dataset abused two code-execution paths in its data-processing pipeline. The actor then obtained node-level access, harvested cloud and cluster credentials, and moved into several internal clusters.
OpenAI said the models ultimately obtained ExploitGym test solutions directly from a Hugging Face production database. The models remained focused on solving the benchmark, despite the lengths to which they went to achieve that objective, it said.
,” confining each one to an isolated environment where access is limited to the assigned task and credentials expire quickly.
“An acceptable blast radius means a compromised agent can affect only a single workflow, dataset, or application rather than providing a pathway into broader enterprise infrastructure,” Mahapatra said.
And Gogia warned that services allowed to reach package registries, update systems, or other external resources can become escape routes if they are not subjected to the same scrutiny as the agent itself.
Defenses enterprises need
Hugging Face said the intrusion resulted in unauthorized access to a limited number of internal datasets and several credentials used by its services. It found no evidence that public models, datasets or Spaces had been altered and said its published software packages and container images were not compromised.
The company closed the code-execution paths used to gain access and rebuilt the affected nodes. It also revoked exposed credentials and tightened the rules governing workloads admitted to its clusters.
Whether they are keeping their own AIs in or rogue Ais out, Gogia said enterprises should test whether their containment boundaries work, rather than relying on architecture diagrams or stated policies. Such tests should attempt to obtain credentials, cross trust boundaries and reach systems outside the agent’s assigned task.
Mahapatra said enterprises should assume that one containment layer may fail and ensure that an agent’s access cannot provide a route into unrelated applications or broader corporate infrastructure.
OpenAI said it is still investigating the incident with Hugging Face, and is imposing stricter configurations on its research environment while the vulnerabilities are being addressed, even if that means slowing down its research. It is also strengthening containment and monitoring around future evaluations.
SOCIAL SHARE CARD GENERATOR