Your Bot’s Next Bottleneck Might Be the Power Grid

What if the hardest constraint on your next agent deployment isn’t context window size, token cost, or inference latency, but whether the local utility can physically deliver electricity to the rack your model is sitting on?

That question stopped being theoretical on April 21, 2026, when Silicon Valley Power, the municipally owned utility of the City of Santa Clara, announced a pilot program with Emerald AI to demonstrate flexible data centers. The goal is to unlock power capacity for AI while keeping performance intact. Emerald AI’s software adjusts data center power consumption in response to grid conditions. NVIDIA is part of the collaboration too, and Emerald AI has announced an integration with NVIDIA DSX Flex.

I build bots. I write about architecture, queues, retries, prompt caching, vector stores. Power grids have never once appeared in my system diagrams. I think that’s about to change, and I think most of us who ship AI systems are underestimating how much.

The layer we never model

Every builder has a mental stack. Mine looks roughly like this: user interface, orchestration logic, model calls, tool execution, storage, monitoring. Somewhere below that sits “the cloud,” which we treat as an infinite, undifferentiated resource that bills us monthly.

The Santa Clara pilot is a reminder that there’s another layer underneath, and it’s made of transformers, substations, and a utility that has to balance supply and demand in real time. When Emerald AI’s software throttles a facility’s power draw in response to grid conditions, something in that facility changes behavior. Maybe it’s batch training jobs. Maybe it’s non-urgent inference. Maybe it’s a scheduler deciding what waits and what doesn’t.

If you’ve ever built a system that degrades gracefully under load, you already understand the pattern. This is the same idea, just pushed down a few layers into physical infrastructure.

Why flexibility beats waiting for more capacity

The conventional answer to power constraints is to build more generation and more transmission. That works, eventually, on a timeline measured in years. The interesting move here is treating the load itself as adjustable rather than fixed.

From a systems design view, that’s the more elegant solution. A data center that can flex its consumption is a participant in grid stability rather than a problem the grid has to absorb. And a utility that can count on that flexibility can plausibly say yes to capacity requests it would otherwise have to defer.

That’s the unlock in “unlock power capacity for AI.” Not new power plants. Better coordination with existing ones.

What this means for how we build

I don’t want to overclaim here. This is a pilot in one city with one utility, and the details of how it performs aren’t public. But the direction it points is worth thinking about now, while our architectures are still young enough to change cheaply.

A few things I’m reconsidering in my own work:

• Job classification. I’ve been lazy about labeling workloads as urgent versus deferrable. If the infrastructure underneath me starts making scheduling decisions, my jobs should carry enough metadata to be scheduled well. A nightly embedding refresh and a live customer-facing agent are not the same thing, and my code should say so.
• Retry and backoff discipline. If capacity becomes something that shifts on a timescale of minutes or hours, aggressive retry loops become actively harmful. Exponential backoff with jitter stops being a politeness convention and becomes basic cooperation.
• Batching as a first-class pattern. Work that can be queued and processed in windows is work that can move to whenever power is plentiful. Streaming-everything architectures give up that option.
• Honest latency budgets. A lot of what we call real-time requirements are habits. Asking which parts genuinely need sub-second response and which parts just felt nicer that way is a useful exercise regardless of grid economics.

The uncomfortable part

None of this shows up in your logs. You won’t see a metric labeled “grid conditions” in your observability dashboard. The abstraction layers that make cloud computing pleasant to use also hide the constraint entirely, right up until it manifests as a region that can’t take new instances or capacity that quietly costs more at certain hours.

That’s the part I find genuinely interesting about Emerald AI’s approach. Software mediating between physical reality and the workloads that don’t want to know about it, which is essentially what every good abstraction does.

Where I land

I’m not going to pretend one Santa Clara pilot rewrites how anyone builds bots tomorrow. It doesn’t. But it makes the physical layer legible in a way it wasn’t before, and the involvement of NVIDIA and Silicon Valley Power suggests the chip vendors and the utilities are both taking it seriously.

For those of us writing the software that consumes all this power, the practical takeaway is modest and unglamorous: build systems that can wait when waiting is fine. That was always good engineering. It’s starting to look like good citizenship too.

🕒 Published: September 18, 2026