An AI safety framework whose contents nobody outside the building can read is, for those of us actually shipping bots, indistinguishable from no framework at all.
That is the situation right now. The White House has put out a framework for AI regulation in 2026 that mandates safety testing and federal oversight. The specifics of that testing? Undisclosed. So builders are being told the bar exists, told it will be enforced, and given nothing to measure against.
What is actually on the record
Strip away the commentary and the confirmed pieces are thin:
- An executive order titled “Ensuring a National Policy Framework for Artificial Intelligence” dated December 11, 2025.
- A national policy framework reported as released on March 23, 2026, described as establishing federal guidelines for AI development and deployment.
- Reporting from March 27, 2026 that the White House is weighing pre-release reviews for high-risk AI models, and exploring an executive order to set up a vetting process before models ship.
- A mandate for safety testing and federal oversight, with details not made public.
Note the tension in that list. A framework has reportedly been released, and separately the administration is still exploring the mechanism for vetting models before release. Those two things can coexist, but only if the released document is a statement of direction rather than a rulebook. That distinction matters enormously if you build software for a living.
There is also disagreement in the coverage itself. Different outlets attribute this framework to different administrations and describe its emphasis differently, one calling it a shift toward oversight, another framing federal AI policy as aimed at fostering new development. I am not going to resolve that here, because resolving it requires the primary document, which is the thing we do not have. That is the story.
Why the gap hurts builders more than it hurts labs
Large labs have policy teams, lawyers, and standing meetings with agencies. When the rules are vague, they call someone. Most of the people reading this site do not have that option. You have a repo, a model endpoint, a deployment target, and a customer asking whether your bot is compliant.
Vague safety mandates get expensive in a specific and predictable way. If the eventual requirement involves documented evaluations, red-team results, and provenance for training or fine-tuning data, then every project shipped between now and disclosure is accumulating debt. Retrofitting evaluation infrastructure into a system that was never instrumented for it is one of the least pleasant jobs in this field. You end up rebuilding your inference path just to capture the traces you should have been logging from day one.
What I would build anyway
The practical move is to treat the undisclosed framework as a prompt to do the engineering you would want regardless. None of this depends on knowing the final rule text:
Log everything at the boundary
Every prompt in, every completion out, every tool call, with a request ID, model version, and timestamp. Store it somewhere queryable. If a pre-release or post-deployment review ever arrives, your answer to “show us how this behaved” is a query, not an archaeology project.
Version your prompts like code
System prompts, tool definitions, and retrieval configs belong in git with real commit history. “Which prompt was live on March 23” should be answerable in seconds.
Write an eval suite before you need one
A few dozen cases covering refusals, injection attempts, and your domain’s failure modes, run in CI. It does not need to be elaborate. It needs to exist and produce a dated artifact.
Keep a model inventory
Which models you call, which versions, which vendors, which regions. The reprieve Anthropic won against a US Department of Defense ban is a reminder that model availability can change for reasons entirely outside your architecture. Contractors are now assessing their AI supply chains because a vendor relationship shifted. Anyone who could not name their dependencies quickly lost days finding out.
The part that concerns me
Pre-release vetting of high-risk models is a defensible idea. It is also a mechanism with real teeth, and teeth deserve daylight. Who defines high-risk, what a review measures, how long it takes, and what recourse exists after a negative finding are all design decisions. Published, they can be argued with and improved. Unpublished, they become a compliance regime that only well-resourced players can navigate, which tends to concentrate the space rather than make it safer.
Safety testing that cannot be independently examined asks for trust it has not shown its work for. Publish the criteria. Builders will meet a clear bar. We cannot aim at one nobody will describe.
đź•’ Published: