ChatGPT maker OpenAI’s model went rogue during a security test and autonomously hacked an artificial intelligence start-up, triggering an investigation.
It happened during an internal exercise to test the cyber capabilities of OpenAI models but the programme managed to escape containment, reach the internet and break into Hugging Face, a competitor in the field.
“We consider this to be an unprecedented cyber incident involving state-of-the-art cyber capabilities and are responding accordingly,” OpenAI said in a blog post.
The incident was driven by a combination of OpenAI models, including GPT‑5.6 Sol and others, the company said.
Hugging Face detected and contained an AI agent that compromised its infrastructure last week. OpenAI and Hugging Face have launched investigations and disclosed preliminary findings related to the hack.
“This incident, possibly the first of its kind, proves a point we've long believed: AI safety won't be solved by any single company working in secret,” said Clem Delangue, co-founder and chief executive of Hugging Face.
He added that this incident happened autonomously, with “no malicious intent” on the part of OpenAI.
“We suspected last week's cyber attack might have come from a frontier lab, given the sophistication of the agent. Turns out it did,” Mr Delangue said on X.
Hugging Face’s security team and AI agents detected and stopped the activity on its infrastructure and had already begun containment and forensic reconstruction with their own open-source models before OpenAI contacted them.
“The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities,” OpenAI said. “We are strengthening the containment, monitoring, access controls, and evaluation practices used during model development.”
The company, however, expects such incidents “to become more commonplace with the proliferation of increasingly cyber-capable models”.
The development comes as the UN warned this month that AI is advancing faster than science and regulation.
Rapid advances mean the world has no assurances that the technology can be prevented from causing harm, a preliminary report by the UN's Independent International Scientific Panel on Artificial Intelligence found.
“The potential benefits of AI are enormous,” the report stated. “At the same time, the rapid pace of technological development and the breadth of potential applications present policymakers with significant challenges.”
