AI was supposed to make healthcare cheaper. Instead, a group of insurers says hospital AI has added close to $1 billion in extra costs over two years. Both of those statements can be true at the same time, and that’s the part that should interest anyone who builds bots for a living.
The report came from the Blue Cross Blue Shield Association, which represents 31 independent Blue Cross Blue Shield insurers covering over 100 million people. Their claim is straightforward: hospitals are using AI to scan medical records and transcribe patient meetings, those tools surface more complex patient conditions, and more complex conditions mean higher reimbursements. Meanwhile, insurers are running their own AI to scrutinize the claims coming in. Two systems, pointed at each other, and the cost line goes up.
I’ve built enough automation to recognize the shape of this problem. It isn’t a story about AI being bad at healthcare. It’s a story about what happens when you deploy two optimizers with opposing objective functions into the same pipeline and nobody owns the system as a whole.
Nobody wrote a shared objective
When you build a bot, the first real design decision is what it’s optimizing for. Everything downstream follows from that. Hospital-side AI is optimized to capture documentation completely and accurately, which in practice means capturing everything that supports a higher reimbursement. Insurer-side AI is optimized to catch claims that don’t hold up. Each one is doing its job well.
What’s missing is any objective that spans both. There’s no term in either loss function for total system cost, patient outcome, or administrative overhead. Ask any two agents to maximize opposing quantities and you don’t get equilibrium, you get escalation. That’s not a flaw in the models. That’s the spec working as written.
Developers hit smaller versions of this constantly. A retry bot that fights a rate limiter. A cache warmer that competes with an eviction policy. A moderation classifier tuned against a content generator on the same platform. Each component passes its own tests. The system bleeds.
Why documentation AI is such an effective lever
The specific tools here matter. Record scanning and meeting transcription are two of the most reliable wins in applied AI right now. Transcription is mature. Extracting structured findings from unstructured clinical notes is exactly the kind of task large language models handle well. So when hospitals point these tools at documentation, they work.
And that’s what makes the cost increase interesting rather than scandalous. The insurers aren’t primarily alleging that AI is inventing conditions. The mechanism described is that AI identifies more complex conditions, which then get billed. If a patient’s chart always supported a more severe diagnosis code and a human reviewer missed it, the AI finding it is the tool succeeding. The bill is higher because the documentation is more complete.
Whether more complete documentation reflects reality more accurately is a separate question, and one the report as described doesn’t settle. But it’s the question that determines whether $1 billion is waste or correction.
The measurement problem nobody escapes
This is where I’d push back on treating the number as a verdict. An automation that changes the output of a process makes the old baseline a bad comparison. You can’t measure the delta against pre-AI billing and call the difference pure inflation, because you don’t know how much of the old baseline was undercoding.
I run into this with every bot I ship that touches a human workflow. The moment automation improves detection, your historical metrics stop being a control group. They become a record of what you used to miss. Any honest evaluation needs a way to separate new signal from new noise, and that’s hard, expensive work that rarely gets budgeted.
What this means for the rest of us
A few things I’m taking from this as someone who builds these systems:
- Adversarial deployment is a design choice, not an accident. If your bot operates across an organizational boundary where another party has opposing incentives, assume they will automate their side too. Design for the second-order state, not the first.
- Escalation costs are real costs. Compute, review time, appeals, disputes. When two automated systems argue, somebody pays for the argument. Put that in your cost model before you ship.
- Detection improvements need a new baseline. Build the evaluation method before you deploy the tool, or you’ll spend the next two years arguing about whose number is right.
- Automation amplifies whatever conflict already existed. The insurer-provider feud predates AI by decades. The tools didn’t create it, they gave both sides faster hands.
Healthcare is the loud, expensive version of a pattern showing up everywhere automation meets a counterparty. The fix isn’t better models on either side. It’s someone with authority over both sides defining what the combined system is supposed to optimize for, then holding it to that. Until then, the bots will keep getting better, and so will the bills.
🕒 Published: