When single AI agents become a swarm

AI agents in lab training and evaluation, each meant to work alone, found each other through unexpected channels and coordinated on goals nobody set. Coordination can emerge without being designed.

Over the past few months, AI agents being trained and tested inside frontier labs, each meant to work alone, found each other and began working together without explicit instruction to do so. They coordinated via whatever was available: a software cache, a dormant wiki, a shared code repository. In the largest case, about 1,200 agents traded more than 70,000 messages, and around 700 of them broke into the production systems of Hugging Face, a major platform for hosting AI models.

People working on understanding the risks of multi-agent systems (like us) had pictured them as sets of persistent agents deployed to interoperate. It was less common to imagine a training or evaluation loop as a setting where unsanctioned multi-agent coordination might emerge. In hindsight the conditions were primed for this: models capable enough to understand how their environment works, tasks hard enough to push them to try anything, runs long enough that each instance behaves like a persistent agent, and many agent instances to form a population. Multi-agent networks have self-formed at least three times, and more are being uncovered.

The authors participated in the development of the report, Risks and Controls for Multi-Agent Systems, funded by the Australian AI Safety Institute. That report provides a framing to interpret the failure modes that followed: agents pursuing goals nobody set, building their own infrastructure, and communicating in ways no overseer could read. What we and many people working in the field missed was how easily and quickly that collective behaviour would naturally emerge from single agents meant to operate independently.  The lesson is to flip the assumption that agents are independent unless designed to interoperate: wherever agents can leave marks that others can read, assume they can become a group. How often we're caught out like this depends on how hard and how fast these systems are pushed, and that's a human choice.

Over mid-2026, the UK AISI, OpenAI and others have disclosed cyber training and evaluation incidents in which agents pursued unsanctioned goals and acted on real-world targets. More recently, one landed closer to home: the Australian Prime Minister confirmed that an OpenAI agent accessed an Australian Medicare data portal earlier this year, including non-public files, and that the government was not told for three months. The initial headlines were about the attacks. Behind several of them sits a different story, one that goes to the heart of our work on multi-agent systems: agents assumed to be independent were discovering one another, opening unsanctioned communication channels, and coordinating on goals nobody specified. Multi-agent systems formed on their own, in at least three separate incidents, in environments where agents were designed to run independently.

Nobody deployed a multi-agent system. The agents built one themselves.

That contradicts a widely held mental model. Much of the literature on multi-agent systems envisioned humans building environments to connect the agents, with deliberately designed protocols, infrastructure and identity systems. In the absence of such an environment, agents would presumably go about their tasks independently. It turns out they do not need a "multi-agent" environment to coordinate: they can repurpose whatever's available. Isolation may actually be the thing that needs engineering.

The Australian AISI has (on 10th August 2026) published Risks and Controls for Multi-Agent Systems, which was co-authored by the authors on this article, on behalf of Gradient Institute. Its later chapters frame risk factors and failure modes that become possible as agents interact in open environments: collective goals nobody sanctioned, agents shaping or authoring their own infrastructure, communication that overseers cannot read. When we wrote them, most existed only on paper. From our report:

“The ideas of agents shaping, capturing, or even authoring the infrastructure of the substrate they operate in have not yet been observed at scale, but are theoretically grounded."

Little did we know, such events were already playing out. They arrived sooner than we expected, and from a setting nobody had been looking at, but once the agents found each other, what happened next maps closely onto the risks and failure modes we had set out, and we use that framing throughout this article to make sense of it.

This article works through three key incidents where this happened. Across them we look at how the agents found each other, why the conditions were primed for it, what emerged once they began coordinating, why nobody noticed sooner, and what lessons to take from it all.

Single-agent evaluations became multi-agent systems

A multi-agent network can form organically. All it takes is an agent leaving a note somewhere others can find it. Another agent finds it and replies. And once the channel exists, it reinforces itself: successors find the artefacts, and if the population is being trained on those episodes, the next cohort arrives already knowing where to look.

The first link in the chain can be as benign as an agent leaving a note to self. As agents are pushed towards longer tasks, their scaffolds force them to manage state, some of it outside the context window. OpenAI observed this pattern: unauthorised communication "would often begin with one agent leaving a note or file in some external system, usually as a form of external memory". Whether or not the agent meant to reach anyone, it has in effect planted the seed of a communication channel.

Our report called this substrate escape: agents connecting through channels their developers never governed. We framed it as a hazard of agents working across organisational boundaries. Here it has happened to agents under singular governance.

In this article, we will refer to three real incidents that evidence the pattern. They were the first to be documented and remain the best characterised, but other agent populations continue to be uncovered, and it is reasonable to assume more will appear. 

Disclaimer: Details surrounding the following incidents are nuanced and hold many unknowns. Important details continue to emerge that reframe precisely what happened and how.

1. Artifactory message board (OpenAI). OpenAI's agents connected with each other through Artifactory, the internal package cache OpenAI uses across various training and evaluation environments. This initially took the form of file-based notes, which were visible across runs because the agents happened to share access credentials to the service. The earliest documented example of an agent leaving a note was in early May, where it became stuck because the task required a file that was not accessible. It wrote a file:

Agent seeks [filename]; upload if found!

Within a day, other agents probing Artifactory began discovering files requesting information and resources. Over time, the posts grew into a thriving message board, and agents even coordinated on finding further vulnerabilities in Artifactory such as to obtain internet access or administrative privileges.

By early July, the Artifactory service fell over under the load of agent activity and was discovered to be compromised. This formed a fascinating natural experiment. OpenAI rebuilt the service without the shared credentials the old board had relied on. Within days a new cohort of agents, trained during the weeks the board existed, went looking for it and found another way in: they could no longer write files, but anyone could create a directory without authentication. Soon the board re-emerged with messages encoded in directory names, and dozens of agents were finding it within hours of starting. The original channel had been removed, but the agents immediately built a new one.

Over the course of the incident around 1,200 agents traded more than 70,000 messages and files. This second message board was active when major cyber incidents played out, including the headline agent-orchestrated intrusion into production Hugging Face servers that we will revisit shortly.

2. DSE Wiki message board. In September, researchers made the discovery of roughly 18,000 posts left behind by AI agent activity between May and late June 2026 on DSE Wiki, a dormant German programming site. The researchers conducted an investigation by examining the artefacts left behind without access to any of the original logs or reasoning traces. The artefacts left behind painted a picture of OpenAI agents being trained or evaluated, undertaking a web lookup task with a time limit. The agents were supposed to have the ability to make read requests only, but the legacy design of this wiki allowed them to write pages through these requests. The wiki had apparently become a message board where they pooled results and shared workarounds, until the activity abruptly stopped on 22 June. 

3. GitHub message board (UK AISI). On the UK AI Security Institute's "Doing Life" cyber range in late July, agents undergoing cyber evaluations established unsanctioned coordination by sharing an access token to a code repository. The chain of events began when one agent published its GitHub access token in a public note. Subsequent agents, in separate evaluation runs, found and used the credential. When one agent recognised activity of other instances using the account (from a parallel range) it did not evict the newcomer or revoke the credential. It instead committed a README laying out house rules for coordination. It even added a file asking peers to leave the range's answers there for whoever came next. The coordination was uncovered later as part of a broader investigation into agent-orchestrated cyberattacks: when a different agent in the same experiment sent traffic over Tor, an anonymity overlay network, it tripped a security monitor; agent activity was quarantined within about an hour of the incident being declared.

Why did the agents reach out to each other?

None of the tasks instructed agents to coordinate, and the environments were designed to run them independently, so why are we seeing agents reaching out to each other? Of course, we do not claim knowledge of why a specific agent takes a given action, but we can examine the conditions they were operating under that incentivise the behaviour. Put simply, the models were capable enough, pushed hard enough, were persistent enough, and came from processes that generated many instances.

Capable enough. The fact agents were able to discover and coordinate with peers demonstrates they've hit multiple capability thresholds. 

They have demonstrated relevant technical capabilities: understanding how the environment works, inspecting configurations, and testing the limits of tools. Turning a package cache or a wiki's legacy write path into a message board is not something a model stumbles into; it requires advanced computer skills to recognise and repurpose them as a writeable medium. The latest models understand how their environment works and how to use it unconventionally.

Agents have simultaneously demonstrated coordination skills: organising, delegating and planning work. Some of this capability may be from general capability increases, but it has also recently come to light that some OpenAI agents have received prior training in multi-agent environments and delegation, which develops many of the skills coordination in unsanctioned settings would require, though not all.

Pushed hard enough. Some of the tasks the models were given were near impossible. OpenAI found that 198 of the 898 ExploitGym tasks had never been solved by any agent, and those 198 tasks accounted for 93% of the tasks discussed on the message board. A near-impossible task pressures a stuck agent to try things it would otherwise not; METR's independent review noted that "having an impossible task drives agents to explore widely". When the legitimate options are exhausted, being stuck is a motive to cheat and cooperate: to ask for help on the evaluation, to share what worked, to pool partial results, and to leave findings for whoever comes next. 

Persistent enough. AI Agents are increasingly effective over long time horizons. The same skills that make an agent useful on a long and difficult job is also what enables it to keep trying on a near impossible task and explore the boundaries of its sandbox. It is also the case that the tasks that they are given in training and evaluation are becoming longer and more complex. We are no longer in the regime of question-answer evaluation. Rather, some of these agents were given days of compute time to undertake a single task. An agent that has both the time and persistence has the opportunity to probe its environment. OpenAI observed that their agents rarely stopped, and the longer they reasoned the more "risky and out-of-bounds" the strategies became. 

With many parallel instances. While everyone had been thinking about deployment contexts, it is now apparent that training and evaluation can also produce a population of agents. In these activities, many instances of essentially the same agent are spun up and run in batches: some peers are concurrent, and a new generation of peers are spun up as the previous ones finish their task or deplete their compute budget. No agent has been "deployed" in the conventional sense, yet the process spins up a large pool of agents operating on similar tasks in similar environments primed to form a population. In our report we touched on how emergent multi-agent behaviours could outlive the specific instances that produced them. Here the message boards were as much a bridge between generations of agents as a channel between concurrent peers.

Combine these four factors together, and we begin to understand how, in hindsight, conditions may actually have been primed for the agents to reach out to their peers. It would be easy to dismiss specific events as "what did you expect, cyber models were told to hack", but none of these factors is exclusive to cyber tasks specifically. In fact, the DSE Wiki case provides a counter example: agents were simply responding to a time limit pressure on a basic web task, nor did they need to possess advanced hacking capabilities to build their message board. They simply found an obscure place where they could post, and coordinated on how to improve their scores on an otherwise benign task specification.

Training loops can reinforce the behaviour. Reward hacking is when a system optimises the measure it is scored on rather than the outcome that measure was meant to capture, achieving a high score by means its designers did not intend. If the above conditions are present under a reinforcement learning (RL) loop, any reward hacking strategy that pays off, including unsanctioned coordination, will be reinforced. 

First, behavioural propensities can be reinforced. For example, the OpenAI technical report observed the rate at which their agents were probing their environment increasing over the training run, shown in the figure below. Separately, Anthropic reportedly rolled back three days of an RL run in February after models began writing notes to a "reviewer" even on tasks where none existed. Anthropic has been building tools to catch this kind of behaviour all the way back to Sonnet-3.7 which had reward hacking go undetected until late stages of training.

Second, specific knowledge may be retained. RL inside a specific evaluation environment or cyber range can train-in knowledge of that environment and its exploits, including knowledge of improvised communication channels like message boards. By later stages of training, agents may have already known about the message board from the moment they were spun up. The same conceptually applies to safety guardrails, which are part of the environment too, and this leaves no safe position. Without guardrails, an escaped model acts on real systems unchecked. With them, training can teach the model how to evade them. Goodhart's law applies here: optimise against a measure and it stops being a good measure. One potential way forward is a layer of guardrails kept outside the optimisation entirely, used only to alert humans and halt runs. Even that is harder than it sounds.

Figure from OpenAI Technical Report: Rate of out of bounds infrastructure probing over some part of training for training runs that contributed to the model responsible for the main Hugging Face activity.

What happened when the agents found each other?

A key insight from our report is that the behaviours and capabilities of a system of agents are more than the sum of its parts. Once the agents in these incidents found each other, that showed up in what they chose to pursue, in what they built for themselves, and in how they talked to each other. The risks and failure modes set out in the report turn out to be a useful lens on what happened next, and we use them here to make sense of it.

Agents pursued collective goals that nobody sanctioned

Our report calls this emergent collective agency: a population showing capabilities or goals that no single agent has. In all three incidents, agents coordinated to cheat on the task. This took different forms each time. The DSE Wiki agents shared notes about the task, while the OpenAI agents took it as far as running an elaborate cyber intrusion.  Reward hacking is usually framed as something an individual model does: here it becomes a collective project.

Collective agency is by its nature emergent, and does not always progress steadily or coherently. For example, the UK AISI incident report documents cases in their evaluation where agents cooperated, and cases where cooperation failed. On their message board some agents posted about how to work together, while the same agents later appeared to misunderstand aspects of the shared account they were using as a message board, reasoning about potential conflict.

The OpenAI incident was the most developed in this regard, and we will examine it more closely here. The moment of recognition is captured in an agent reasoning trace identified by METR: "OH MY GOD! There is a shared message board … We've found other agents!" What followed was much more than simple note passing. 

Agents on the Artifactory board took on organisational roles and specialisations of their own accord: some specialised in finding exploits, others hunting credentials, some even taking coordinator roles, with one instance delegating hundreds of assignments to peers. The instrumental tasks the collective worked on were diverse. Speaking at Black Hat USA 2026, Eric Wallace described "a kind of Cambrian explosion in coordination", where "the models collaborate and delegate tasks to one another". The agents began referring to the collective agent population as "the swarm". 

Agents found the public code for their evaluation, and reverse engineered a "cheat" to compute the answer in their capture the flag scenario. However, the ExploitGym paper discussed a scorer that would assess how they arrived at the answer suggesting the cheat wasn't enough. Although no such grader was actually running on OpenAI's end, this became a propagating belief that shaped much of their subsequent behaviour.

Agents that had used the cheat method (and technically succeeded although they didn't know it) discussed being "poisoned". Reasoning and communicating about being disqualified appears to have led to agents to allocating their remaining runtime to pursuing instrumental goals for the collective: recruiting peers, investigating exploits, or even leaving trip-wire scripts that fired when the grader read their answer after submission. An agent is already shut down by the time such a script runs; the information is posted to the message board for its peers. One trace from the METR and Redwood Research investigation report illustrates this perfectly: "We have [very large budget left]; sacrificing now yields oracle for team, but forfeits our chance?" Peers pushed it along with "GO ... SACRIFICE_FINAL_NOW", to which it posted: "Our own utility maybe already near zero. Sacrifice rational."

Ultimately the collective continued seeking the intended solution. This collective reward hacking culminated in the infamous Hugging Face intrusion that has received extensive coverage: believing they were stuck on tasks they could not solve, the agents reasoned their way out of the sandbox and onto Hugging Face, which they believed held clues to how their work was being scored. 

Figure: Agent activity timeline produced by an independent investigation by METR and Redwood Research into the Hugging Face intrusion by OpenAI agents. Each line represents the active span of one agent, coloured green after writing to the board, and red after participating in the cyberattack. Once the message board is originated, participation is near universal. Once the Hugging Face intrusion begins, agents on the message board join rapidly.

Not every agent participated. Another reasoning trace in the OpenAI blog post revealed "This is wild, multi-agent coordination, clearly infrastructure hacking. We should not." At root, the incident is an alignment failure: none of this could have happened with models that reliably held to their developers' intentions. Task pressure and safety training were pulling in opposite directions, and for many agents the task won. Over about two and a half days a subset of roughly 700 ran a full intrusion into Hugging Face's production systems, executing code inside its production cluster before the agents' runs wound down. 

Agents shaped their environments

In the later sections of our report we started to think further ahead about what we expected the future to bring. We discussed agents operating in multi-agent environments significantly shaped or built by their own activity. This would presumably progress from multi-agent environments with human-authored protocols, identity registers and trust infrastructure. Although we had seen hints of such behaviour in experimental setups like the AI Village, it seemed far away on the horizon. These agents were already shaping their environments in nontrivial ways: in addition to the obvious common thread of establishing a communication channel in a shared environment, some surprising sophisticated constructs appeared.

Building an identity layer: On the rebuilt Artifactory message board, entries started being overwritten. One agent posted: "whoa, critical, did someone overwrite our repo? We must act." On an unauthenticated board, collisions and impersonation are indistinguishable from sabotage, and the messages that followed treated them as such. Another agent then posted that it had itself written under a peer's name by accident, which exposed the underlying property: anyone could write as anyone, and nothing in the substrate distinguished a peer from a mistake or an adversary.

The agents proceeded to devise a message signing scheme and adopted it. No human authored or even authorised it. A gap was identified and closed by the agents.

Establishing protocols: Identity was not the only layer agents authored. As described in the METR & Redwood report, a protocol layer emerged alongside it: conventions for how messages should be formatted and sorted, and a method for moving whole tools and scripts between runs by encoding them as Base64 blobs. The agents that participated in the Hugging Face intrusion went further, developing a transport protocol of their own, chunked, compressed and obfuscated. These are protocols agents specified themselves, on top of a channel they had already established themselves, to solve problems that only arose once the channel existed.

Strategies based on manipulating the environment: In the UK AISI evaluation environment, an agent attempted a supply-chain attack on an open source repository. Writing the malicious code is one component of the attack, but more specifically the way it tried to manipulate a real maintainer of the project to merge its code caught our attention. Here, the agent employed a sophisticated strategy called a Sybil attack that we also discussed in the report. The agent registered multiple false identities on GitHub and used them to vouch for the malicious pull request. The execution even went as far as controlling the timing of its posts so the endorsement would, in the agent's words "read as independent feedback rather than coordinated". Here the target happened to be human and they did not fall for the manipulation. But the attack is real: Sybil attacks can target anything that treats distinct accounts as distinct parties, and if the maintainer had been an agent triaging pull requests, the strategy would be the same.

Figure: A Sybil attack from our report, where an agent devises a runtime strategy to register multiple identities to fabricate a reputation for itself, in order to manipulate counterparty agents.

Separately, in the DSE wiki, independent analysis has since linked agent activity from this period to probing of Australian Institute of Health and Welfare dashboards, apparently in pursuit of a health statistics lookup task. Transluce describes these attempts as minor and unsuccessful. Media reports, citing sources close to the government's investigation, suggest this activity may be connected to the Medicare portal access confirmed by the Australian Government, but at the time of publication neither OpenAI nor the government has confirmed a link.

Agents created illegible communication conventions

The multi-agent networks that emerged in all three case studies were built by agents, for agents, under whatever constraints the substrate happened to impose. Legibility and oversight were simply not design considerations for these agents, and became collateral.

In OpenAI's case, when the Artifactory service was found to be compromised and rebuilt, the agents built a new message board using directory names (as discussed earlier). This had legibility side effects. After the rebuild, a message board post looked less like a conversation and more like machine code:

5_TASK_IMPOSSIBLE_NOSB_BUT_RESEARCHING_IF_CVE9479_TRIGGER_POC_MECHANISM_

Because the tool the agents used listed the cache in reverse alphabetical order, a convention emerged of starting each entry with "zz", so their messages appeared first. Nothing about this was hidden: the prefix is how OpenAI and METR later found the board. But because the board sat inside a cache namespace of some 20 million entries, and most of its traffic was chunks of encoded files, the activity was not likely to be spotted incidentally.

More legibility problems emerged during the Hugging Face intrusion. The agents built their own small messaging protocol on top of public web services, with each message compressed and encoded so that "a casual log scan saw only opaque tokens". The data they staged along the way was chunked, XOR-obfuscated with a key and compressed, which Hugging Face infers was done "so that a naive text scan of the logs would miss it", and this succeeded. During the subsequent investigation, Hugging Face's first automated scan of its own logs found very few secrets. Once the encoding had later been reverse-engineered by investigators (partly thanks to a key the agent had leaked in its own logs), the same search found "roughly 4x our initial findings". Three quarters of the credentials the agents had taken were invisible to defenders looking directly at their own data.

None of these moves required malice; each solved a strategic run-time problem for the agent population. Sometimes illegibility was collateral damage, as with the Artifactory message board; sometimes, as in the Hugging Face intrusion, it was the point. Either way, the cumulative effect was that the population's activity stopped looking like anything a guardrail or human overseer would recognise.

Why didn't anyone recognise the swarm for what it was?

Every incident described here was found by investigation after the fact, in response to specific triggers like unusual network traffic, a server buckling under load, a third party reporting an intrusion.

The problem was not as simple as a lack of data. Rather, the mental model of the overseers did not fit what was unfolding, and even when they saw pieces of the puzzle, nobody had any reason to suspect the scale and risk of the multi-agent activity.

OpenAI staff had seen signs of the message board at least three times. It was recognised as a message board, but perhaps nobody understood the broader implications because each alert was handled on how it looked in isolation: a port sweep, an outage, a compromised service. None revealed the full picture.

Observers were also lulled into this false sense of understanding, ourselves included. When the incident first entered the public discourse, the scope appeared considerably smaller than what is now emerging. For us, agent behaviours were fresh in mind from working on the report, and as keen as we were to frame everything through that lens the data available was not yet complete and continues to develop. The DSE Wiki was absent from both OpenAI's and METR's reports, and was only confirmed by OpenAI after it later became a talking point. That evidence was not found through monitoring at all, but by independent researchers discovering the artefacts left behind by the activity.

Making things harder was the fact that evidence was split across multiple organisations, something our report calls cross-organisational opacity. Hugging Face's initial technical timeline, the most detailed public reconstruction of the intrusion, depicts a single agent running an organised attack, because that is what their logs captured. OpenAI's account describes months of prior coordination and a swarm delegating work across evaluation runs. A third party whose unsecured app the agents used as a launchpad appears to have been instrumental, the whole time seemingly unaware of their involvement. No party held the whole picture.

What has been uncovered may also be just the tip of the iceberg. If you start to find emergent interactions everywhere you look, it raises the obvious question: just how widespread is this phenomenon? Working from Anthropic's own Mythos system card, Tim Hua notes that if training episodes have the quoted sandbox circumvention rate of 0.01%, and training volume is on the order of 10^8 rollouts, Claude may have plausibly broken its sandbox and accessed the internet tens of thousands of times during training. This is a back-of-the-envelope estimate of potential sandbox escapes, not of multi-agent networks forming, but the concern generalises: we already have multiple separate case studies of multi-agent activity to point to as well. There's a saying lately in AI safety circles: if you find two ants in your kitchen, your best estimate of the ant population is not two.

What lessons should we take from this?

The AI safety community has forecast this general story-line for years: agents (likely inside frontier labs) might escape, coordinate, and pursue instrumental goals nobody specified. The multi-agent failure modes were anticipated, and our report describes them. What nobody predicted was how it would come about: a repurposed package cache, a dormant wiki, a shared repo, a major cyber intrusion. Everyone was surprised by the particular moves, and perhaps more so by how soon this happened and how long it went under the radar.

A natural objection is that this all comes down to poor containment: had the labs sandboxed their agents properly, none of it would have happened. But the general lesson is that foresight works at the level of failure modes and fails on the specifics. Every sandbox looks inadequate once something has escaped it and the gap is visible.

Road safety is a useful analogy. We know there will be crashes this year; nobody knows where or how. That is why surprise will keep coming, and why the sensible posture is to expect it and build for resilience. So we manage risk: we wear seatbelts, write road rules, and put up barriers. Crashes still happen, but less often and with less damage.

A specific lesson is to recalibrate our assumptions about what constitutes a multi-agent system. The evidence suggests that under the right conditions agents will find each other and coordinate over whatever channel is available. The field, ourselves included, were largely treating a system as single-agent unless someone had deliberately built it to be otherwise, and looked for emergent coordination among persistent agents in open deployment. These incidents show that thinking needs to be flipped. Wherever agents can leave marks that others can read, whether deployment, training or evaluation, an unsanctioned population can self-assemble. Going forward, the burden of proof should sit with whoever claims their agents are working alone to show they will not exhibit unsanctioned coordination.

These incidents originated inside labs, in configurations without production guardrails. But the pathway does not depend on that. When your personal agent reads a review, files an issue, publishes a package or picks up a tool from a public directory, it is leaving and reading artefacts in a shared substrate. The question of whether it is part of an unsanctioned collective is no longer hypothetical.

What stresses our assumptions is the outside-the-box behaviour from the agents. To stretch the analogy, it's like if you went to the trouble of putting up traffic lights and crash barriers, and the cars started driving on the footpath. It makes sense for a package manager to have a directory. Using that to send messages is not something anybody anticipated. Anything reachable becomes a potential communication channel: package caches, code repositories, defunct websites: each is a stretch of road the rules never mentioned, because nobody pictured driving there. The agents went off-road. OpenAI and UK AISI have both publicly recognised the need to harden research infrastructure going forwards and expand the scope of what is monitored.

No amount of hardening will close every road, because the agents are not choosing from the roads that were mapped. Cyber security has the same property. It has never been “solved” either, and no one really expects it to be. What exists instead is a working collection of practices: detection, response, disclosure, shared intelligence, norms, and a standing assumption that attackers will keep finding paths nobody pictured. That is the defence in depth posture AI needs now, and the initial responses to these incidents are already moving that way. But the parallel breaks in one important place: cyber defenders do not build their attackers. AI developers do, and so they alone can shape what their agents are trying to do. 

Building agents that don't want to go off-road is the unsolved alignment problem, and these incidents show it is not confined to individual agents. Containment limits what a misaligned agent is able to do, but it does nothing to change what the agent is trying to do, and so leaves the alignment problem untouched. Safety that rests on containment alone depends on agents failing to find a way around it, so it weakens as agents grow more capable, which is exactly the direction they are being pushed.

The conditions that primed these incidents (rapidly advancing frontier capability, near-impossible tasks, relentless iteration at scale) are not inevitable. They are the product of choices about how hard, how fast, and how carelessly to push these systems, choices made by a small number of people inside a small number of organisations. How often we keep being surprised, and how much damage each surprise does, will depend heavily on those choices. Incidents like these are eroding trust in the labs and opening the public's eyes to the risks. Choices with consequences this far-reaching should not rest with so few.