Here is a claim that will annoy half the people reading benchmark charts this year: the transistor count on an AI accelerator is one of the least interesting things about it. We obsess over the logic die like it’s the whole product. It isn’t. It’s a passenger. The parts that decide whether your training run finishes or your inference endpoint falls over are the ones nobody puts on a slide — the memory stacks, the interposer, and the unglamorous slab of resin and copper underneath it all.
I build bots for a living. That means I spend an embarrassing amount of time waiting for tokens, watching batch sizes bump into memory ceilings, and reading vendor spec sheets that tell me almost nothing about why one accelerator serves my model at half the latency of another with similar paper compute. The answer, most of the time, is not the math units. It’s what surrounds them.
The chip you never hear about
Modern accelerators are not chips in the way we used to mean it. Designers now place multiple large logic dies next to a stack of memory on a single package. That whole assembly sits on an ABF substrate — the layered organic package that routes thousands of connections between the silicon and the board. It’s a supply chain choke point, it’s hard to make, and it’s the reason “we taped out a great die” doesn’t guarantee “we shipped a great accelerator.”
Meta’s third-generation MTIA is a good illustration of where the industry’s attention actually goes. Full specs aren’t public, but what is known points at TSMC’s N3P process, very likely over 100 billion transistors, and HBM memory. Two of those three facts are about how the thing is built and fed, not about the architecture of the compute itself. That ratio is not an accident.
Follow the money, and it points at memory
The funding pattern tells the same story. In February 2026, Positron AI raised $230M in a Series B for a memory-centric inference accelerator. Read that description again. Not compute-centric. Memory-centric. Somebody wrote a nine-figure check on the thesis that the bottleneck in serving models is moving weights, not multiplying them.
The same month, Cerebras closed a $1,000M Series H for its wafer-scale training and inference processor. Cerebras’ entire pitch is a packaging argument dressed up as a compute argument — keep the work on one enormous piece of silicon so you stop paying the tax of shuffling data between packages. WSE-3 is now a real competitor to NVIDIA alongside AMD’s MI300X+, and its differentiator has never primarily been the arithmetic.
What the market share numbers actually show
NVIDIA still holds roughly 80–85% of data center AI accelerator revenue in 2026, down from about 92% in 2024. AMD sits around 5–7%, up from roughly 2%. Google’s TPUs hold their own slice. Intel’s Gaudi 3 underperforms, and Broadcom’s custom XPUs keep gaining traction.
That last pairing is the interesting one for anyone building on this hardware. Gaudi 3 is not failing because Intel forgot how to design multipliers. Broadcom isn’t winning custom XPU sockets because it invented better math. Broadcom wins on the parts of the problem that live at the package and system level — getting memory bandwidth and interconnect right for a specific customer’s workload. The buyers placing those orders are hyperscalers who measure tokens per dollar per watt at rack scale, and at that scale packaging decisions dominate.
Why a bot builder should care
You can’t buy an interposer, so what do you do with any of this? A few things I’ve found practical:
- Treat memory capacity and bandwidth as your primary spec. When comparing instance types, sort by HBM capacity and bandwidth before you look at FLOPS. Your KV cache lives there, and your context length is a memory question.
- Profile data movement, not just kernel time. If your inference path is memory-bound, buying more compute buys you nothing. Measure it before you upgrade.
- Expect supply weirdness to hit availability, not just price. Advanced substrates and memory stacks are the constrained inputs. When a specific instance type is unobtainable for a quarter, that’s usually why. Write your serving layer so it can move between accelerator families.
- Don’t assume the market leader’s part is the right part. With AMD taking real share and custom silicon growing, a solid abstraction over your inference backend is worth the engineering time.
The competitive story of 2026 gets told as a race between architectures. It reads more like a race in assembly — who can put the most memory closest to the most logic on a package that doesn’t warp, and who can get enough of those packages built. That’s a manufacturing contest with a compute scoreboard.
Next time you’re comparing accelerators for a bot deployment, spend less time on the headline chip and more on everything crowded around it. That’s where your latency actually comes from.
🕒 Published: