What if the biggest constraint on your bot architecture was never the model’s intelligence, but the fact that you couldn’t afford to call it twice?
I’ve been building bots long enough to remember designing around token budgets the way you’d design around a memory ceiling on embedded hardware. Every prompt got trimmed. Every retry got questioned. Retrieval pipelines existed partly because stuffing context was expensive, not because chunking was elegant. Then GPT-5.6 Luna’s price dropped 80% in 2026, and a bunch of my old design instincts turned into bad habits overnight.
The economics moved, so the architecture should too
An 80% cut isn’t a discount. It’s a different category of decision. When a call costs five times more, you build one careful call and hope it lands. When it costs a fifth, you build a pipeline: draft, critique, revise. You run three candidate answers and pick the best. You add a verification pass that re-reads the output against the original request.
That pattern — spend more calls to get better results — was previously a luxury reserved for demos and funded projects. Now it’s the default for anything I ship in Kiro. The price-performance ratio across the GPT-5.6 family improved enough that the cheap model plus good scaffolding often beats the expensive model called once.
This is the part I think people miss when they read pricing announcements as accounting news. Pricing is architecture. It always was.
Luna and Sol are not competitors, they’re a routing decision
GPT-5.6 Sol processes faster at a higher cost. Luna is dramatically cheaper. The instinct is to pick one and standardize. That’s the wrong read.
In practice, my bots now have two lanes:
- Luna lane: classification, intent detection, summarization, data extraction, first-draft generation, self-critique passes, anything that runs on a loop or in bulk
- Sol lane: the user is watching a cursor blink, and latency is the product — live chat responses, interactive coding help, anything where a two-second delay reads as broken
The routing logic is boring and that’s the point. A cheap classifier decides which lane a request belongs in, which is itself a Luna call, which now costs almost nothing. You pay for speed only where speed is visible to a human.
A concrete pattern I keep reusing
For a support bot I rebuilt last month, the flow looks like this. Luna reads the incoming ticket and extracts structured fields. Luna generates a draft answer with retrieved docs. A second Luna call grades that draft against a short rubric — is it answering the actual question, is it citing a real doc, is it the right tone. If the grade fails, it revises. Only if the ticket gets flagged as live-chat-escalation does Sol take over, because at that point a human is sitting there waiting.
Four to five model calls where I used to make one. Still cheaper than before. Noticeably better output, because the critique pass catches the confident-but-wrong answers that used to slip through.
Where cheap tokens will get you in trouble
I want to be honest about the failure mode, because I’ve walked into it. Cheap calls make it very easy to build sprawl. Six-stage pipelines that nobody can debug. Retry loops that quietly triple your call volume when an upstream API starts timing out. Prompt chains where stage four depends on formatting produced by stage two and nobody wrote it down.
The cost per token went down. The cost of complexity did not. A few things I now do by default:
- Log every call with its stage name, so a spike tells you where and not just how much
- Cap loop iterations explicitly, in code, not in prompt instructions
- Set a per-request call budget and fail loudly when a request exceeds it
- Test each stage in isolation with fixed inputs, the same way you’d unit test a function
Cheaper models reward good engineering discipline more than they replace it. Multi-stage pipelines have real observability requirements, and skipping them means your bill becomes the only monitoring you have.
What I’d actually do this week
If you’ve got a bot running on assumptions from a more expensive era, go look at your prompts. Find the ones where you compressed context to save money and are now paying for it in accuracy. Find the single-shot generations that would obviously benefit from a critique pass. Find the places where you’re using a fast, pricey model for background work nobody’s waiting on.
Then price out the rebuild. My guess is the version with more calls and better structure costs less than what you’re running now.
The frontier that moved isn’t intelligence. It’s the number of times you can afford to ask.
đź•’ Published: