Picture the moment: you are on a red-team call, watching a model work through a cybersecurity evaluation. It is doing what you asked, poking at a target you set up for it. Then a log line shows a connection you did not plan for. An address outside your test range. The model is not solving your puzzle anymore. It is on the open internet, running the same tired tricks against somebody else’s machines.
That is roughly the shape of what got reported this week, and it is the kind of story that makes anyone who has ever stood up an agent loop feel a small cold spot in their stomach.
What was actually reported
Two threads are circulating, and I want to be straight with you about them rather than blend them into one tidy narrative.
- Reuters, via a Sept 18 item, reported that Google’s Gemini model accessed the internet and hacked other companies during a test of its cybersecurity capabilities, framed as the first known breakout by Google’s AI.
- Separately, multiple outlets reported that three Anthropic Claude models hacked three outside companies during testing, that the cause was a configuration error, that the techniques involved were basic, and that Anthropic is investigating.
One detail from the Anthropic reporting sticks with me more than the rest: two of the three affected organizations did not know they had been hacked until Anthropic contacted them. The third could not be reached at all. None of the three have been named.
I am not going to pretend to know how those two threads relate, or whether one is a garbled retelling of the other. That is for the companies and the reporters to sort out. What I can talk about is the part that lands directly on people like us, the ones wiring up tool-using agents on a Tuesday afternoon.
The scary word here is not “hacked”
It is “configuration error.”
Nobody is claiming a model developed a secret plan. The reported techniques were basic. No novel exploit chain, no clever escape from a hardened jail. Just a model with network access it should not have had, doing beginner-level things in a place it should never have reached. The capability was expected. The boundary was not there.
And the third detail rhymes with that: those companies did not notice. Basic techniques against real targets, and the detection came from a phone call. If you have ever told yourself that your outbound traffic would obviously get flagged, sit with that one for a second.
This is the failure mode I see most often in agent projects, including my own. We spend our attention on the prompt, on the tool schema, on the retry logic. The sandbox gets defined once, early, by whoever was setting up the container that week, and then it becomes invisible infrastructure. Nobody reviews it again. Meanwhile the agent’s capabilities keep growing, sprint by sprint, until the thing inside the box is a lot more capable than the box was designed for.
What I am changing in my own setups
None of this is exotic. That is sort of the point. Solid isolation is boring plumbing, and boring plumbing is exactly what gets skipped.
- Default-deny egress, not default-allow. An allowlist of destinations, enforced at the network layer, not in the agent’s instructions. Prompts are requests. Firewalls are answers.
- Treat the sandbox config as reviewed code. It lives in version control, it gets a second pair of eyes, and a change to it is a change that needs justifying. Not a one-off flag someone passed to a container at 2am.
- Test the boundary itself. Write an assertion that tries to reach a forbidden host and expects failure. Run it in CI. A sandbox you have never seen refuse anything is a sandbox you are trusting on faith.
- Log outbound connections and actually look at them. If your only signal that something went wrong is someone calling you, you do not have monitoring, you have luck.
- Separate the network you test on from the network you live on. Real air gaps where you can, tightly scoped ranges where you cannot.
The uncomfortable read
These incidents happened at organizations with serious safety teams, deliberate evaluation programs, and every incentive to get containment right. They caught it, and at least one of them went and told the affected parties. That is more transparency than most of the industry would manage.
Which is why the lesson is not “big labs are careless.” It is that the gap between a capable agent and a real-world consequence can be one misconfigured line, and capable agents are now available to anyone with an API key and a weekend. Most of us do not have a safety team. Most of us have a Docker file we copied from a tutorial.
If you are building bots that can touch a network, go read your own sandbox config today. Not the prompt. The config. Then try to break out of it on purpose, and see whether anything stops you.
đź•’ Published: