Remember when the whole pitch for building software was that you didn’t have to think about hardware? Serverless, elastic, infinite. You wrote a function, pushed it, and the machine underneath became somebody else’s problem. A lot of us built entire careers inside that abstraction. I know I did.
That abstraction is leaking. Fox News reported this week that one of America’s oldest manufacturers is on a path to double in size because of the AI boom, according to its CEO. Frontieras North America is publicly repositioning coal for what it’s calling the AI economy. Consumer Reports is running pieces on what data centers do to electric bills and water. CNET is writing about AI data centers coming for land, power, and water. Kai Williams put together 16 charts at understandingai.org just to explain the shape of the boom.
None of that is a software story. All of it is now our story.
The abstraction was always a loan
I build bots. Retrieval pipelines, tool-calling agents, the usual stack of prompts and vector stores and retry logic. For most of that work, the physical world showed up as one number on a dashboard: cost per thousand tokens. That number was the only signal I ever needed about the turbines, transformers, cooling loops, and industrial supply chains sitting underneath my API calls.
What the current news cycle makes clear is that the signal was always incomplete. When a manufacturer with a century-plus of history says it’s doubling because of AI demand, that’s a physical footprint expanding to meet demand that people like me generate, one agent loop at a time. When old energy sources get repackaged for data center load, that’s a downstream consequence of architecture decisions made in a text editor.
We’ve seen a smaller version of this before. During the crypto mining years, GPU prices went sideways and hobbyists who just wanted to train a small model got priced out of their own workbenches. The lesson then was that compute demand is not abstract. It shows up in retail shelves, in power bills, in local politics. That lesson is back at a much larger scale.
What this changes about how I build
I’m not going to pretend one journalist’s CEO quote should rewrite your entire stack. But it does shift some engineering defaults that were only ever defensible when compute felt free.
- Model sizing is now a design constraint, not an afterthought. Reaching for the largest available model on every call is the equivalent of running a space heater to warm a coffee cup. Route by task difficulty. Small model first, escalate on failure.
- Caching is an ethics feature, not just a latency feature. Every deduplicated request is inference that never happens. Semantic caching on repeated user intents is one of the highest-return things you can add to a bot, and it always was.
- Agent loops need hard ceilings. An unbounded ReAct loop that “keeps thinking” is a bug that bills you and the grid. Cap iterations. Log when the cap gets hit. Treat runaway loops as incidents.
- Batch what doesn’t need to be live. Overnight embedding jobs, nightly summarization, backfills. Off-peak scheduling is a real lever, and it’s a solid habit even when nobody’s watching.
- Measure tokens the way you measure memory. If you can’t answer how many tokens your bot spends per resolved user request, you don’t actually know your system.
The uncomfortable part
I want to be careful here, because the easy move is to turn this into a guilt trip aimed at developers, and that would be wrong. The energy and water questions Consumer Reports and CNET are raising get decided by utilities, regulators, siting boards, and the companies writing the checks for these facilities. My caching strategy is not the deciding variable.
But the ratio matters. If your bot burns a hundred model calls to accomplish what twelve would have done, you’re not the problem, you’re a rounding error in the problem, multiplied by everyone else shipping the same way. Efficient systems are also cheaper systems, faster systems, and systems that degrade more gracefully when capacity gets tight. There’s no tradeoff to argue about.
Where the interesting work is
What genuinely excites me about the physical turn in this story is that it makes engineering craft valuable again. When compute felt unlimited, the winning move was to throw more of it at every problem. When compute has a visible cost in factories, land, and electric bills, the winning move goes back to what it used to be: knowing your data, sizing your models, writing tight retrieval, and shipping something that does the job without waste.
That’s the kind of building I want to keep writing about here. Not because the grid needs me to. Because that’s the version of this craft that’s actually worth learning.
🕒 Published: