It’s 2:14 in the morning and your fine-tuning job has been sitting in a queue for forty minutes. The provider dashboard says “capacity constrained, retrying.” You refresh. Still queued. You downgrade to a smaller instance type, cut the batch size, and tell yourself you’ll just check the loss curve in the morning. Somewhere in that same moment, a hyperscaler is signing a purchase order large enough to make your entire year of compute spend a rounding error.
That gap between those two moments is basically the whole story behind Jensen Huang’s latest number.
Speaking at Goldman Sachs’ Communacopia + Technology Conference, Huang repeated a claim he’s now said more than once: Nvidia can grow revenue 70% year over year. “I think we could grow 70% year over year. We’re confident about that,” he said Thursday. Analysts expect Nvidia to close its current fiscal year around $400 billion in revenue. Apply 70% to that and you land near $680 billion. Huang’s reasoning is straightforward — Nvidia’s position in AI computing plus demand for its products that keeps outrunning what it can supply.
Why a bot builder should care about someone else’s earnings call
I don’t own Nvidia stock and I’m guessing most of you reading a tutorial site don’t either. But that 70% figure is not really a finance story for us. It’s a capacity forecast, and capacity forecasts are the thing that decides whether your inference bill next quarter looks anything like this quarter’s.
Huang also pointed to supply constraints as part of the picture. That’s the part that lands directly on our desks. When demand exceeds supply at the chip level, the shortage doesn’t stay at the chip level. It travels down through cloud providers, then through the GPU rental marketplaces, then to the person trying to grab an eight-GPU node on a Tuesday afternoon. Big customers get their allocation first. Small builders get the leftovers, at leftover-market prices.
What to actually do about it
Growth predictions are not something you can act on. But planning for a compute market that stays tight for another couple of years absolutely is. A few things I’ve changed in how I build:
- Stop treating one GPU type as a hard requirement. If your training script only works on a specific card with a specific driver and a specific attention kernel, you’ve handed your schedule to whoever controls that inventory. Test on at least two hardware targets, even if one is slower.
- Make checkpointing non-optional. Spot and preemptible capacity is where the actual availability is. That’s only usable if losing an instance mid-run costs you fifteen minutes instead of a day.
- Measure your inference cost per request, not per month. A monthly total tells you nothing when unit prices move. Per-request numbers let you see a price change the week it happens instead of at the end of the billing cycle.
- Push the small model harder than feels comfortable. A quantized 7B model that runs on hardware you can reliably get beats a larger one you’re constantly waiting in line for. I’ve lost more time to queues than to weak model quality.
- Cache like it’s a feature, not an optimization. Prompt caching, embedding reuse, memoized tool calls. Every request you don’t send is a request that can’t get rate-limited.
The uncomfortable read
There’s a version of this news that’s genuinely good for us. Demand that high means more capacity gets built, and built capacity eventually becomes cheap capacity. The same pattern played out with general-purpose cloud compute — scarce and expensive, then boring and commoditized. If Huang’s number is right, a lot of silicon is heading into data centers, and some of it will eventually be available to people building side-project bots at 2 AM.
There’s also a less fun version. A company projecting 70% growth on a $400 billion base is describing a market where the buyers with the deepest pockets set the terms. In that market, indie builders are not the customer being optimized for. Pricing, allocation, and product roadmaps get shaped around the people writing nine-figure checks.
Both versions can be true at once, and probably are. Which is why I’d rather build for the constrained case and be pleasantly surprised. Architecture that survives scarce, expensive compute also runs beautifully on abundant, cheap compute. The reverse is not true.
So take the 70% for what it’s useful as: a signal that the compute crunch you’ve been feeling in your build pipeline is not a temporary glitch in your provider’s scheduler. It’s the shape of the market for a while. Design accordingly, keep your stack portable, and stop assuming the GPU you used last month will be there next month.
🕒 Published: