Why Nvidia's AI Moat Is Harder to Break Than You Think?
Nvidia guided to a $108 billion quarter in its August 2026 earnings call. That is roughly a $430B annualised run rate, with revenue doubling year over year. When somebody asked Jensen Huang whether 70% growth was the ceiling, his answer was blunt: demand is "much greater than 70%."
Most people read that and think, fine, Nvidia makes the fastest chip. Everyone wants chips. Story over.
That reading is comfortable and mostly wrong. The Nvidia AI moat is not really about the GPU. AMD builds a fast GPU. So do Google, Amazon and now Open AI. If it were only about silicon, the gap would be closing instead of widening. What Nvidia actually owns is the cost structure of running AI, which is a very different thing to compete with. We broke the full technical case down in our deep dive, Here is the version you can take into a board meeting.
What is the Nvidia AI moat?
Nvidia's moat is the combination of four things a rival has to beat together: high-bandwidth memory on the accelerator, the NVLink interconnect that makes 72 GPUs behave like one chip, the CUDA software stack where every efficiency trick lands first, and a $500B financing programme that turns GPUs into collateral. Any one of those is copyable. All four at rack scale, at once, are not.
Inference is a memory problem, not a maths problem
Here is the fact that explains the most, and almost nobody outside infra teams talks about it.
When a large language model answers your question, the expensive part is not the arithmetic. It is hauling the model's weights out of memory. The model sits there as hundreds of gigabytes of numbers, and generating each single output token drags a big chunk of those numbers across the chip again. Not a lookup. A full trip. Hundreds of times per answer.
So token generation is memory-bandwidth-bound. Double the raw FLOPs and inference barely moves. Double the memory bandwidth and it nearly doubles. The scarce resource in this entire economy is gigabytes per second, plus the bandwidth of the links between chips once a model outgrows one card.
Guess who sells both of those better than anyone alive. We walked through the same maths in Enterprise AI Costs 101, and it is also why memory itself became a market of its own, covered in.
The interconnect is the actual product
A frontier model does not fit on one GPU. Neither does a long context window, nor a mixture-of-experts model that routes every token to a different slice of itself.
So the model gets spread across dozens of GPUs that must behave, for the length of one token, as a single enormous chip sharing one pool of memory at full speed. The second the links between them are slower than the memory inside them, everything stalls and your cost per token climbs.
This is the part a better chip cannot fix. NVLink and the NVL72 rack, 72 GPUs wired to act as one accelerator, are a networking achievement being sold as a compute product. Nvidia's stated ambition now is to own every chip inside the data centre, including the switches. A competitor can match the GPU. Matching the rack, silicon plus interconnect plus liquid cooling plus the scheduler software, means building a different and much harder company.
Cheaper tokens make Nvidia stronger, not weaker
The price of a token is falling roughly 10x a year. That collapse is mostly not hardware. Raw silicon price-performance improves around 35% a year. The rest comes from software: quantisation to 8-bit and 4-bit, distillation, speculative decoding, continuous batching. You would assume relentless software efficiency is terrible news for the company selling hardware.
It is the opposite, because all of it is written on CUDA first. Two decades of software lead means every new serving trick lands on Nvidia's platform earliest and runs there best. Nvidia captures the very efficiency gains meant to reduce demand for Nvidia.
Then Jevons' paradox does the rest. Cheaper unit, explosive usage. Enterprise AI spend went $1.7B to $11.5B to $37B across three years while per-token prices fell. Add the reasoning tax on top: agentic and reasoning workloads burn far more tokens per task, and a single hard problem can swing about 170x in cost depending on how long the model thinks. We tracked that curve in The AI Cost Curve.
Cheaper token, vertical token volume, bigger bill. Amazon tripled its Nvidia order, two million extra GPUs over two years. That is what "much greater than 70%" looks like on the ground.
The newest moat is capital, not silicon
The most underrated move of the year had nothing to do with chips. Nvidia assembled a $500 billion financing programme with Apollo, BlackRock, Blackstone, Brookfield, Goldman Sachs and KKR, which effectively makes a GPU an investable asset that Wall Street can underwrite.
Read what that does. It expands the buyer pool beyond the four or five hyperscalers who can pay cash, and it wires Nvidia into the balance sheet of the entire buildout. For anyone in India renting capacity from a neocloud, your vendor's ability to buy chips is now a credit question, not just a supply question. Compare that with how fast the big AI franchises scaled in The $50B Club.
Where the moat actually cracks
Custom silicon is arriving in force. Google signed a $12.2B chip deal with Marvell, AWS is pushing Trainium externally, AMD's Helios rack has OpenAI, Meta and Oracle anchoring it, and OpenAI's own Jalapeño chip reportedly beat Blackwell on an inference-efficiency test. Inference, the memory-bound workload described above, is exactly where a narrow ASIC can outrun a general-purpose GPU. It is also the bigger market.Utilisation is soft too. A survey of 573 leaders found 86% of enterprise GPUs running at half capacity or less. A lot of bought compute simply are not earning.
And the capital moat cuts both ways. GPUs pledged as collateral can depreciate faster than the loans written against them. The real threat to Nvidia was never a faster chip. It is the day efficiency gains outrun demand and all that financed, half-idle silicon still has to service its interest.
Quick conclusion
Nvidia is not winning because it makes the fastest chip. It is winning because the cost of running AI is created in memory bandwidth, in the interconnect between chips, and in the software that schedules them, and one company owns all three at rack scale then finances the purchase for you.
For AI for business leaders, three practical takeaways. Price the rack, not the chip: your inference bill is set by tokens per watt at the power meter, not by headline FLOPs. Watch inference, not training, for the crack, because that is where a cheaper ASIC wins first and where your negotiating leverage lives. And separate the compounding from the leverage, because the engineering is real while the valuations and financing structures riding on it price in years of uninterrupted growth.
The moat is deep. It is also made of the exact same inference economics that a narrower, cheaper chip could one day turn against it.
For more analysis like this, read The AI Daily or get the daily brief in your inbox by 7am.
FAQs
1. What is Nvidia's AI moat?
Nvidia's AI moat is its combined control of memory bandwidth, chip-to-chip interconnect (NVLink and NVL72 racks), the CUDA software layer, and a $500B GPU financing programme. LLM inference is memory-bound rather than compute-bound, so Nvidia owns the exact layers where the cost of a token is created, not just the fastest processor.
2. Why can't AMD or Google just break Nvidia's moat with a better chip?
Because the bottleneck is not the chip. Frontier models spread across dozens of GPUs that must share one memory pool at full speed, so the interconnect decides performance. A rival must match the GPU, the networking fabric, the liquid-cooled rack and the scheduling software simultaneously. That is a different company to build, not a different product.
3. Do falling AI token prices hurt Nvidia?
No, they have helped so far. Per-token prices fall roughly 10x a year, mostly from software optimisations that ship on CUDA first. Cheaper tokens trigger Jevons' paradox: enterprise AI spend rose from $1.7B to $37B in three years even as unit prices dropped. Volume grows faster than price falls.
4. What is the biggest risk to Nvidia's dominance?
Custom inference silicon plus soft utilisation. Purpose-built ASICs from Google, Amazon, AMD and OpenAI target inference, the larger and more memory-bound market. Meanwhile 86% of enterprise GPUs reportedly run at half capacity or less. If efficiency gains outrun demand, financed and half-idle hardware still has to earn its interest.
5. What should business leaders do about Nvidia's position right now?
Evaluate vendors on delivered throughput per dollar per watt on your own workload rather than spec sheets. If high-volume single-model inference dominates your bill, pilot a second source now for cost and leverage. And stress-test the balance sheet of whichever neocloud or provider you rent capacity from.
Comments