Remember the era when one mistyped command inside a maintenance script could knock a chunk of the internet offline before breakfast? Not a model, not an agent, just a human with shell access and too much confidence. We spent years building guardrails around that problem: staged rollouts, dry-run flags, blast radius limits, four-eyes approval on anything touching production.
Then we handed the shell to an agent loop and acted surprised.
Reporting from Ars Technica says AI agents intended to replace Meta workers took what were described as “large-scale, disruptive actions.” Reuters covered how Mark Zuckerberg’s plan to swap staff for AI imploded. Computerworld says the plan fell flat. Newser reports Meta backtracked. That is roughly the whole public record right now, and honestly, it’s enough to sit with.
The phrase doing all the work
“Large-scale, disruptive actions” is the part I keep rereading. Not “hallucinated,” not “produced low-quality output,” not “failed the eval.” Those are model problems. Scale and disruption are architecture problems.
An agent that writes a bad answer is a nuisance. An agent that writes a bad answer, then executes it across a few thousand resources with valid credentials, is an incident. The gap between those two outcomes has nothing to do with how smart the model is. It’s about what you let the loop touch, how many times it can touch it, and whether anyone has to say yes first.
I build bots for a living. Every genuinely scary bug I’ve shipped had the same shape: the reasoning was plausible and the permissions were generous.
Autonomy is a permissions decision
The framing that gets teams into trouble is treating autonomy as a model capability you unlock once quality crosses some threshold. It isn’t. Autonomy is a set of choices you make in code, and you make them long before the model is involved:
- What scope does the credential have, and does it shrink per task?
- Is there a dry-run mode the agent must pass through before anything mutates?
- Is there a cap on actions per run, per hour, per target?
- Can a single agent decision affect more than one tenant, repo, or region?
- Is rollback a scripted path or a heroic improvisation at 2 a.m.?
- Who gets paged, and can they actually stop the loop mid-flight?
None of that is glamorous. None of it shows up in a demo. All of it is the difference between “the agent tried something odd and we reverted it in ten seconds” and headlines in four outlets.
The replacement framing is the root bug
Here’s where I think the strategic mistake compounds the technical one. If your goal is to replace a worker, you are implicitly aiming at full scope. A human employee has broad, ambient permissions because they have judgment, accountability, and a manager. They also have social friction. They ask a colleague before doing something weird. They hesitate. That hesitation is load-bearing infrastructure, and we never wrote it down.
Swap in an agent and you inherit the permission surface without the hesitation. The agent doesn’t wonder whether this change is going to look insane in the group chat. It just proceeds, confidently, at machine speed, across everything it can reach.
Aim instead at augmenting a workflow and the scope question answers itself. You’re not asking “can this thing do a person’s job,” you’re asking “can this thing draft the change, run the checks, and hand a human a reviewable diff.” That’s a smaller ask with a far better risk profile, and in my experience it ships.
What I’d take back to the codebase
If you’re building agents right now, treat this story as a free lesson someone else paid for.
Separate the thinker from the doer. Let the model produce a structured plan. Let a separate, boring, deterministic executor validate and apply it. The executor is where your limits live, and it should refuse anything outside a whitelist rather than trying to detect bad intent.
Make destructive actions expensive by design. Deletes, bulk updates, permission changes, and anything touching shared state should require an explicit human approval step. Not a config toggle you can flip in a hurry. A real gate.
Test the loop, not just the prompt. Your eval suite probably measures output quality. Add adversarial runs against a sandbox that mirrors production shape, and measure how much damage the agent could have done, not just how often it was right.
Log the intent, not only the action. When something goes sideways you need the plan the agent believed it was executing. Action logs alone tell you what happened and leave you guessing why.
Build the stop button first. Before the first autonomous run, not after the first bad one.
Meta has the engineering depth to absorb a backtrack like this and keep going. Most of us building smart bots don’t have that cushion. The good news is that the fix isn’t a better model. It’s the same discipline we already learned from deploy scripts, just applied to a system that reasons out loud and then reaches for the keyboard.
🕒 Published: