My grandfather owned a table saw, a router, and a set of chisels. He used the chisels most. Not because they were more powerful, but because most of the work in front of him was small, specific, and needed a tool he could hold in one hand without clearing the garage first. The table saw was for the days when you actually needed a table saw.
I think about that a lot when I’m building bots. For roughly three years the default move has been to reach for the biggest model available for every single task, including tasks that amount to sorting text into three buckets. That habit is starting to break. The AFR reports that Australia is leading a rise in small language models, which tells you the shift isn’t just a hobbyist thing happening in someone’s homelab. It’s showing up as a national research and industry story.
Why the timing makes sense
Two threads in the recent news feel connected, even though they come from different corners.
The first is MIT Technology Review’s reporting that AI’s recursive self-improvement might not arrive as quickly as promised. If you were building your roadmap around the assumption that models would soon bootstrap themselves into something dramatically more capable, that assumption just got more expensive to hold. Which means the practical question shifts from “what will the frontier do next year” to “what can I ship with what exists right now, at a cost I can actually defend.”
The second is a hardware rumor, of all things. AppleInsider is reporting that the 2027 iPhone 18 may ship with 12GB or 9GB of RAM and a 2nm A20 chip. I’m not in the business of predicting Apple’s roadmap, but as a builder I read RAM specs the way a chef reads oven size. More on-device memory and more efficient silicon is the substrate small models run on. If that’s where phones are heading, the addressable space for models that fit in a few gigabytes gets a lot larger.
What this changes in practice
Here is where my opinion gets specific, because I’ve been restructuring how I build.
Most bot architectures I see are one big model doing everything: intent detection, entity extraction, routing, tone matching, generation, and safety filtering, all in one call with a prompt that’s grown to 4,000 words of accumulated instructions nobody wants to touch anymore. It works. It’s also slow, expensive, and impossible to debug, because every failure looks like the same failure.
Small models let you go back to something more like a pipeline:
- Classification and routing handled by a small local model that responds in milliseconds and costs nothing per call
- Extraction and structured output handled by a small model fine-tuned on your actual data format, which usually beats a general model prompted at it
- Generation escalated to a larger model only when the task genuinely needs breadth
- Safety and formatting checks as separate small steps you can test in isolation
Each stage becomes something you can write a unit test for. That alone changed my week more than any model upgrade did.
The cost math nobody puts in the demo
A demo bot handling 200 requests a day costs nothing meaningful no matter what you run. A bot handling 200,000 requests a day is a budget line item somebody will eventually ask you to justify. When you break the work into stages, you discover most of your volume is cheap work wearing an expensive coat. Route it accordingly and the bill drops without the user noticing anything except lower latency.
Where this gets interesting geopolitically
Rest of World has a piece on Americans choosing Chinese AI, and I find that story relevant here for a builder-specific reason. When capable models become plentiful and cheap enough to run in more places, model choice stops being a one-vendor decision and starts being a supply question: what’s available, what’s licensed how, what can I run where my data is allowed to live. Small models make that portability real, because a model you can actually host is a model you can actually move.
That’s a different kind of freedom than the API era gave us. Not better on every axis. Just different, and worth designing for.
My take
Small models aren’t a downgrade dressed up as a trend. They’re a correction. We spent a while confusing model size with engineering quality, and the projects I’m proudest of lately are the ones where I picked something small, fine-tuned it on a narrow job, and let it be excellent at exactly one thing.
Reach for the chisel first. Keep the table saw plugged in for when you need it.
🕒 Published: