Remember the regex era of content moderation? If you built anything with a user-generated input field between roughly 2010 and 2018, you probably shipped a blocklist. A flat file of banned words, maybe a few clever patterns to catch character substitutions, and a cron job to review the flags. It worked right up until the moment someone realized they could just… not use the banned words. Then you learned the hard lesson every filter-builder learns eventually: adversaries don’t walk into your pattern match. They walk around it.
Meta announced Wednesday that it is rolling out new AI tools aimed at a particularly nasty version of that walk-around problem — ads that appear harmless on the surface but function as directions to child sexual abuse material. The same announcement included a number worth sitting with: the company took action against 33.2 million pieces of child sexual exploitation content across Facebook and Instagram in the first half of 2026.
Signposting is an intent problem, not a content problem
The technical term Meta is using is “signposting,” and it’s a good description of why this category is so hard to catch. A signposting ad doesn’t contain the violating material. It contains a pointer. A phrase that means nothing to a classifier and everything to someone who already knows the code. A profile handle. A platform name spelled oddly. An emoji sequence that has drifted into a shared vocabulary somewhere off-platform.
If you’ve built classifiers, you know why this wrecks the standard approach. Supervised text classification is good at surface features: tokens, phrases, patterns that correlate with labels in your training set. Signposting has almost no stable surface features by design. The signal lives in context, in co-occurrence, in the gap between what the text says and what it is clearly trying to accomplish.
Meta’s answer is a large language model tasked with identifying those tactics. That makes architectural sense. An LLM can reason over the full ad — copy, creative, destination, surrounding account signals — and answer a question closer to “what is this trying to get me to do?” rather than “does this string appear in my list?” It’s the difference between pattern matching and intent inference, and for adversarial coded language, intent inference is the only approach that has a chance.
The part builders should steal is the red-teaming agent
The second piece of the announcement is the one I keep thinking about for my own projects: a red-teaming AI agent whose job is to test Meta’s own safety measures. Not a human review team. Not a quarterly audit. An agent that probes the defenses.
That’s a pattern worth internalizing, because most of us build safety systems backwards. We write the filter, we test it against the examples we thought of, we ship it, and then we wait for production to tell us what we missed. Production tells us by failing. The failure is the test.
An adversarial agent in the loop flips that order. You generate attempted bypasses before deployment and keep generating them after, so your evaluation set grows at roughly the same rate the real-world evasion tactics do. For a static threat, a fixed test suite is fine. For an adaptive adversary, a fixed test suite is a snapshot of a problem that has already moved.
How to apply this at smaller scale
You don’t need Meta’s infrastructure to use the shape of this design. A few things translate directly:
- Separate detection from evaluation. Two different systems with two different objectives. The detector tries to catch things. The red-teamer tries to slip things past the detector. Don’t let one model grade its own homework.
- Log near-misses, not just blocks. The content that scored just below your threshold is your best source of signal about where the boundary actually sits.
- Treat your eval set as a living asset. Version it. Grow it. Every bypass your red-teamer finds becomes a permanent regression test.
- Give the model more than the text. Destination, account history, timing, creative. Intent inference needs context, and a text-only input starves it.
- Keep humans on the escalation path. An LLM making a judgment call about intent is making a probabilistic guess, and the cost of being wrong in either direction here is real.
What I can’t tell you
I’d want to know a lot more than the announcement provides — precision and recall figures, how the red-teaming agent is scoped, how appeals work when the model misreads a legitimate ad. Those details aren’t public, and I’m not going to guess at them. What’s visible is the architecture, and the architecture is sound: reasoning models for coded intent, adversarial agents for continuous pressure-testing, and a scale of enforcement activity that makes clear why manual review alone was never going to be enough.
Most of us are building bots with far lower stakes than this. The design lesson still holds. If your safety system has never been attacked by something you built on purpose, you don’t actually know how solid it is.
🕒 Published: