Blue Cross Blue Shield’s position, stripped of the press-release polish, is this: the data backs up the claim that AI is driving up medical bills. Not someday. Now. And when I read that, my first reaction wasn’t outrage at hospitals or sympathy for insurers. It was recognition. Because I have shipped bots that did exactly this kind of thing, and I know precisely how it happens.
It happens because the bot did what you asked.
Nobody wrote a cheating function
Here’s the mechanic insurers are describing. Clinical documentation tools read the encounter, the notes, the dictation, and produce a more complete record than a tired human would at 7pm on a Thursday. More complete documentation supports higher complexity codes. Higher complexity codes mean bigger bills. Bigger bills, aggregated across a payer’s book of business, mean higher premiums.
Notice what is missing from that chain: fraud. There is no line of code that says if profitable: inflate(). The objective was “capture everything that happened in this visit.” The system met the objective. The billing consequence was downstream and, to the people building the tool, arguably invisible.
Any of us who have built agents recognize this shape. You give a bot a metric, it optimizes the metric, and the metric turns out to be a proxy for something you actually cared about. Documentation completeness is a proxy for quality of care. Billing codes are a proxy for documentation completeness. Chain two proxies together, point a capable model at the first one, and the second one drifts.
The part that should make builders uncomfortable
I keep coming back to the framing in the 2026 healthcare cost trend analysis I’ve been reading through: AI documentation and coding tools will likely accelerate an existing trend around more complex patient populations. Accelerate. That word is doing a lot of work.
The trend already existed. Coding creep is older than transformers. What AI did was remove the friction that had been acting as an accidental brake. Thorough documentation used to be expensive in human attention, so it happened inconsistently. Now it is cheap and consistent. The brake is gone.
That is the general lesson, and it applies far outside healthcare. When you automate a process that had a natural cost ceiling, you do not just make it faster. You change its equilibrium. Every bot I have built that touched a workflow with financial consequences eventually surfaced this question, usually after launch, usually in a meeting nobody enjoyed.
Bots on both sides of the desk
The genuinely strange part is where this goes next. CBS News reported in March on U.S. hospitals and insurers both turning to AI in fights over claims and payments. Providers use models to document and code. Payers use models for prior authorization, which is the approval gate providers have to clear before treatment. That’s a documented use, laid out in the 2026 Kansas legislative briefing on AI in health insurance.
So: a model generates the claim, another model evaluates the claim, and the humans involved are increasingly reviewing outputs rather than making decisions. Insurers are adjusting policies to manage the spending, which is a polite way of saying they are tuning their side of the loop in response to the provider side of the loop.
As an architecture, this is adversarial optimization with patients as the substrate. If you have ever watched two automated systems negotiate with each other in production, you know they do not converge on fairness. They converge on whatever the scoring rules reward, and they get there fast.
What I actually take from this
I am not going to pretend I have a policy fix. I build bots. But there are things I have changed about how I build them, and this story is a good argument for all of them.
- Trace the second-order metric. If your bot’s output feeds a system with money attached, instrument that system too. Documentation quality looked fine. Billing totals were the tell.
- Treat capability increases as policy changes. Making an expensive process cheap is not a neutral efficiency win. It shifts behavior at scale, and you should predict the shift before shipping.
- Assume an adversary is automating too. Design for the state where the other side of your transaction is also a model. Single-sided optimization assumptions break immediately.
- Log enough to answer “why this output.” When someone eventually asks whether your system nudged a code upward, “the model decided” is not an answer anyone will accept.
The insurers may be partly self-interested here. Higher premiums are their story to tell and their bill to send. But the technical claim underneath is plausible, and I have not seen a good counterargument to the causal chain itself.
The uncomfortable version of this story for those of us who build things is that no villain is required. Competent engineers built useful tools that worked, and the cost showed up somewhere the engineers were not looking. That failure mode does not announce itself. You have to go find it.
🕒 Published: