Experts keep warning that risks from unchecked AI development are escalating. Fine. I believe them. But what caught my attention this week isn’t the warning itself — it’s the silence that follows whenever someone asks the frontier labs a very simple question: if one of your models went rogue, how exactly would you contain it?
The labs remain tight-lipped about their containment strategies, and that bothers me more than any doomsday forecast. I build bots for a living. Small ones, mostly — customer service agents, scraping pipelines, automation glue. And I can tell you the first thing any competent builder designs is the off switch. Not the personality. Not the prompt chain. The kill path.
Containment is an engineering problem, and engineers publish
When my team ships an agent, we can answer the containment question in one breath: revoke the API key, drop the container, cut the network egress rule, done. It’s boring. It’s documented. Anyone on the team can execute it at 3 a.m. without waking me up.
So when organizations with a thousand times my resources won’t describe their equivalent, I see two possibilities, and neither is comforting:
- They have a plan but won’t share it. Maybe for security reasons, maybe for competitive ones. But secrecy around safety mechanisms is backwards. In every other engineering discipline — aviation, nuclear, medicine — safety procedures are the most public part of the system. Scrutiny makes them stronger.
- The plan is thinner than we’d hope. Containing a small bot is easy because its capabilities are narrow and its infrastructure footprint is known. A frontier model with broad capabilities, wide deployment, and API access threaded through thousands of downstream products is a genuinely harder shutdown problem. If the honest answer is “we’re still working on it,” I’d rather hear that.
Regulators are asking my question, just slower
Regulatory efforts are now underway to mandate shutdown mechanisms for these systems. On one hand, good — someone is finally treating the off switch as a requirement rather than a nice-to-have. On the other hand, it says something uncomfortable that it took legislation to force the conversation. Mandated kill switches are the seatbelt laws of AI: obvious in hindsight, resisted in the moment.
My worry, as a builder, is that a legally mandated shutdown mechanism becomes a checkbox rather than a tested capability. There’s a huge difference between “we have a shutdown procedure written down” and “we ran the shutdown drill last Tuesday and it worked.” Every bot builder knows this. Untested rollback paths fail exactly when you need them. If regulators mandate the mechanism, they should mandate the fire drill too.
What a real answer would look like
I’m not asking the labs to publish attack surfaces. I’m asking for the kind of answer any of us would demand from a junior engineer in a design review:
- What are the layers? Model-level, infrastructure-level, network-level?
- Who has authority to pull the trigger, and how fast can they act?
- What happens to downstream deployments when the switch flips?
- When was the procedure last exercised, and what broke?
None of that reveals secrets. All of it builds trust. The current posture — assurances without specifics — asks the public to extend the kind of faith we stopped extending to bridge builders a century ago. We inspect bridges. We should inspect off switches.
The small-scale lesson scales up
Here’s what a decade of shipping agents has taught me: the systems that hurt you are never the ones you planned for. They’re the ones where a permission was scoped too wide, a credential lived too long, a loop ran without a circuit breaker. Containment isn’t one heroic mechanism. It’s a hundred boring constraints, applied early, tested often.
If that’s true for my little bots, it’s true a thousandfold for frontier models. The escalating-risk warnings from experts land differently when you’ve personally watched a misconfigured agent
🕒 Published: