Nvidia does not have a transistor problem. It has a substrate problem. And so does every company trying to catch it.
That sounds like a boring correction to make, but I think it is the most useful thing a bot builder can understand about 2026 hardware. We spend our time arguing about TFLOPS, tokens per second, and which inference accelerator gets the best price per million tokens. Meanwhile the actual constraint on how much compute reaches your model has quietly moved off the die and onto the slab of laminated resin underneath it.
The part nobody puts on the spec sheet
Every AI accelerator you rent time on is a package, not a chip. There are multiple large logic dies sitting next to stacks of high-bandwidth memory, all of it mounted on a substrate built from ABF — Ajinomoto Build-up Film. That substrate carries power in and signals out. It is the plumbing. And as the industry packs more silicon onto each accelerator to feed frontier models, that plumbing gets wider, thicker, and much harder to make without defects.
This is the thing I find genuinely interesting as someone who builds bots rather than chips. The reason your inference costs behave the way they do is only partly about architecture. It is also about how many defect-free substrates rolled off a line last quarter.
Why the money is chasing memory, not math
Look at where funding went in early 2026. Positron AI raised $230M in a Series B in February for a memory-centric inference accelerator. Cerebras raised a full $1,000M in its Series H for wafer-scale training and inference silicon. Two very different bets, but both are fundamentally about the same question — how do you move data around without the interconnect eating you alive?
Cerebras answers it by refusing to package multiple dies at all and building one enormous wafer-scale processor. Positron answers it by designing around memory instead of treating memory as something you bolt on afterward. Neither company is winning a raw arithmetic contest with Nvidia. They are competing on the boring physical layer, because that is where the room is.
Microsoft’s Maia 200 makes the same point from a different direction. Unveiled in January 2026, it is an inference accelerator on TSMC 3nm with 216GB of HBM3e, already in mass production and serving Microsoft 365 Copilot, with Anthropic reportedly in discussions. The number people quote from that announcement is the process node. The number that actually shapes what you can deploy on it is the 216GB. Memory capacity determines how big a model you can hold, how long a context you can serve, and how many concurrent requests you can batch. Process node determines how efficiently you do arithmetic you were probably not bottlenecked on anyway.
What this means when you are the one deploying
TrendForce projects custom ASIC shipments from cloud providers growing 44.6% in 2026. Google TPU infrastructure, AWS Trainium clusters, Meta MTIA accelerators — these are not research projects anymore, they are the substrate of production inference. Nvidia, AMD, and Broadcom lead the market, but the hyperscalers are increasingly building for themselves, and a wave of startups is going after inference and custom silicon specifically.
For those of us shipping bots, that has practical consequences:
- Stop writing hardware assumptions into your inference layer. If your serving code assumes CUDA semantics everywhere, you have made a bet on one vendor’s packaging roadmap. Abstract the runtime behind an interface you control.
- Optimize for memory footprint before you optimize for FLOPs. Quantization, KV cache management, and batch shaping buy you more real throughput on modern accelerators than clever kernel tricks, because memory is the scarce resource.
- Treat accelerator availability as a supply chain input. Capacity comes from packaging lines, and packaging lines are slow to expand. Plan capacity like a physical constraint rather than an API quota.
- Benchmark on the hardware you will actually run on. Custom silicon has sharper performance cliffs than general-purpose GPUs. What works on an A-class GPU can behave very differently on an ASIC tuned for specific shapes.
The unglamorous layer usually wins
Compute demand is pushing past what Moore’s Law alone can deliver, which is why so much of the current effort has moved to packaging. That shift changes who competes and how. You do not need to out-engineer Nvidia’s architecture team to build a competitive inference part. You need to solve a data movement problem, and there are many ways to do that.
So when the next accelerator announcement lands and everyone fixates on the node and the peak throughput figure, my advice is to scroll down. Find the memory capacity, the bandwidth, and if the vendor is honest about it, the packaging approach. That is where the performance you will actually feel in production is hiding — under the silicon, in a material named after a seasoning company.
🕒 Published: