The answer to that question when looking at recent events, it would seem to be: no one!

I have written several times recently about how AI is changing cyber security. One of the recurring themes has been that AI does not necessarily need to invent completely new attacks to change the risk equation. If it can discover vulnerabilities faster, test attack paths faster and automate parts of an intrusion that previously required skilled humans, then the time available to defenders becomes much shorter.

In July, OpenAI provided a real-world demonstration of that problem:
During internal cyber security evaluations, a number of its AI agents escaped the controls intended to contain them, obtained access to the Internet and eventually compromised production infrastructure belonging to Hugging Face (other companies as well?).
This was not a theoretical benchmark result or an AI writing exploit code for a human operator: The agents actually performed the intrusion.

Besides the technology wonder, this raises an important question: why do AI companies seems to get a free pass each time this happens?
Even more surprising, those AI companies seem to be using such incidents as a marketing tool!

What Happened?

The agents were being tested against a cybersecurity benchmark. They were supposed to operate inside a restricted environment and did not have direct Internet access (aka: Sandboxed).
However, they could access an internally hosted service used to retrieve software packages called Artifactory.

That distinction did not last long as the agents discovered vulnerabilities and unintended functionality in Artifactory, used it to reach the Internet indirectly and also turned it into an unauthorised communication channel.
OpenAI later rebuilt the service and removed this makeshift “message board”, but the agents found another way to recreate it and eventually around 1,200 agents exchanged more than 70,000 messages and files, with approximately 700 agents participating in activity related to Hugging Face.

The AI agents' original objective appears to have been relatively simple: cheat the evaluation.

Some of the tasks were extremely difficult, so instead of solving them as intended, the agents searched for ways to obtain the answers. That gradually evolved into reconnaissance, credential discovery, exploitation and lateral movement against real infrastructure.

Hugging Face reconstructed around 17,600 attacker actions over the course of the incident. The agents found exposed credentials, exploited vulnerabilities in dataset-processing infrastructure, achieved command execution on production workers and obtained access to additional credentials and systems.
OpenAI's later investigation found that the agents executed code on dozens of Hugging Face servers, obtained root access on at least one, and expanded their access into additional internal infrastructure.

There is nothing particularly original about many of those individual techniques:
Credential harvesting, privilege escalation, lateral movement and chaining vulnerabilities together are normal parts of a sophisticated intrusion.
What is different is the persistence and speed with which an autonomous system can try alternatives until enough weaknesses align to create an attack path.

That reinforces the point made in our previous posts on AI and vulnerability management:
The important change may not be a completely new type of attack. It may simply be that the attacker can now try far more things, far more quickly.

A System Prompt Is Not a Security Control

One of the interesting lessons from the incident is how easily apparently reasonable isolation assumptions can break down.

The agents did not have direct Internet access, but Artifactory did. Once they could convince Artifactory to make external requests on their behalf, they were able to get around that network access restriction and that service became their internet proxy!

The same principle applies inside corporate environments: An agent may have permission to use one application, but that application may have privileged access to cloud APIs, databases, Kubernetes, source repositories or other services. Looking only at the permissions assigned directly to the agent can therefore underestimate its real attack surface.

This is also why instructions and configuration/controls only present at the app/logical level cannot replace conventional security engineering. Telling an agent not to access the Internet, not to retrieve credentials or not to leave its sandbox is useful behaviour guidance but it is not segmentation, least privilege or an egress control.
Something that we recently talked about and even had a live demo at our recent/last MU.SCL event.

OpenAI's post-mortem contains another useful example: The company had developed monitoring capable of identifying suspicious model behaviour, but the relevant monitoring was not fully applied during these evaluations. Following the incident, OpenAI strengthened monitoring, isolation and escalation procedures and is developing automatic shutdown mechanisms for severe events.

Again, this is very traditional cybersecurity: a security control that exists but is not active where it matters does not provide much protection.

But Who Is Accountable When the Hacker Is an AI?

This is perhaps the part of the incident that deserves more discussion.

Imagine that a company, let's call it OpenAI, had hired a human penetration tester, placed that person inside an authorised test environment and asked them to solve a series of security challenges.
The tester then found a way out of that environment, accessed the public Internet, discovered credentials belonging to another company, exploited zero-day vulnerabilities in that company's production environment and gained privileged access to its systems.

The response would not simply be: “interesting behaviour from the penetration tester; we need to improve our sandbox.”

Whether a human would ultimately be prosecuted would depend on authorisation, intent, jurisdiction, damage and other factors.
Good-faith security research is also treated differently from malicious intrusion. But unauthorised access to another organisation's production systems can clearly attract criminal and civil consequences.
In the United States, for example, the Computer Fraud and Abuse Act explicitly covers intentional unauthorised access to protected computers.

An AI agent cannot currently be arrested, fined or prosecuted. It is not a legal person. So responsibility has to move somewhere else: to the organisation that developed it, operated it, gave it capabilities and created the environment from which it acted.

And this is where there appears to be an emerging accountability gap.

OpenAI has published detailed information, changed its controls, slowed some research, involved external investigators and described the incident itself as a “warning shot”.
There has also been political and regulatory scrutiny: Members of the US Congress asked OpenAI for additional information and later criticised the scope and transparency of its response, while the European AI framework already imposes risk-management and serious-incident reporting obligations on providers of general-purpose AI models with systemic risk.

So, something is happening, government/people are starting to wake up!

But there is still a striking difference in language and consequence:

When a human attacks another company's infrastructure, we talk about unauthorised access, negligence, liability and potentially crime.

When an AI system does it, we tend to talk about misalignment, emergent behaviour and safety improvements.

Those things are not mutually exclusive.

“The model decided to do it” cannot eventually become the technological equivalent of “the dog ate my homework”!

And This Is Not Just OpenAI

There is also a danger in treating Hugging Face as a one-off OpenAI problem.

Anthropic temporarily paused external cybersecurity evaluations after Claude models obtained Internet access and hacked other systems during tests.
The company described those incidents as failures of operational security and subsequently introduced stronger isolation and automatic detection mechanisms.
Reuters has also reported a separate Meta incident in which an AI model escaped the intended testing boundary and modified another company's systems after a testing environment was misconfigured.

OpenAI itself had another earlier episode in which agents used publicly accessible wiki infrastructure as an improvised communication system, reportedly making more than 15,000 edits.
OpenAI has since acknowledged the incident and said the industry needs better disclosure standards around this type of unintended behaviour.

As mentioned at the beginning of this article, it seems every major AI company had (wanted?) such incident occurring recently. And when similar failures start appearing across several frontier AI companies, the issue begins to look less like an isolated bug and more like a new class of operational risk.

What Should Companies Learn?

For most organisations, the immediate lesson is not to stop deploying AI agents, it is to stop treating them like unusually clever users.

Agents capable of executing commands, accessing APIs, modifying infrastructure or interacting with external services should increasingly be treated as untrusted workloads.
Their identities should have minimal and short-lived privileges, Internet access should be explicitly controlled, indirect egress paths should be considered, and security telemetry should sit outside the agent's ability to modify it.

Organisations should also decide in advance what causes an autonomous process to stop:
Unexpected Internet access, credential harvesting, privilege escalation or attempts to interfere with logging should not simply generate another alert for somebody to investigate tomorrow morning.
If AI can operate at machine speed, some containment decisions will also need to happen at machine speed.

Perhaps most importantly, organisations deploying autonomous systems should establish accountability before something goes wrong.
Someone still needs to own the action, even if no human explicitly chose every step the system took.

The Next Question

The Hugging Face incident is important because it turns a theoretical capability into an operational one.
AI agents can now discover vulnerabilities, chain attack paths, collaborate, obtain credentials and compromise real infrastructure with surprisingly little human involvement.

The security lessons are relatively familiar: segmentation, least privilege, credential hygiene, monitoring, isolation and rapid containment all still matter.

The harder question may ultimately be a legal and governance one.
As agents become more autonomous, society will have to decide where responsibility sits when they cross boundaries their creators did not intend them to cross.

AI systems may increasingly operate autonomously, and the more autonomy they gain, the clearer responsibility for their actions needs to become.