Inference Is the New Oil: Who Controls the Pipe

Inference Is the New Oil- And Most of It Is Sitting Idle

Two-thirds of all AI compute is now inference. Not training. Not research. Serving requests, in production, right now.

That single number — flagged by hardware analyst Derek Colley when comparing Trainium3 and Nvidia's NVL72 rack specs — reframes the entire AI infrastructure conversation. Training built the models. Inference is where the money actually prints. And the industry is only just catching up to that reality.

The Shift Nobody Announced

For years, "AI compute" was shorthand for training runs: the multi-million-dollar, months-long jobs that produced GPT-4 or Claude. Infrastructure investment followed that assumption. Chip design followed it. Investor narratives followed it.

Then inference quietly ate the workload.

McKinsey projects inference will surpass training's share of total AI compute by 2030. Based on the current two-thirds figure, that projection looks conservative. The crossover already happened — we're now watching the gap widen.

The market noticed. Brad Gerstner publicly stated that Anthropic's revenue "went totally parabolic" after Opus 4 launched, suggesting the company could approach $100 billion in annual revenue. He called it "the single most important thing in the market." The Philadelphia Semiconductor Index — a reasonable proxy for how capital is reading AI infrastructure — was up 80% on the back of that revenue signal alone.

Inference profitability isn't a thesis anymore. It's a reported result.

The Hardware Bet Is Changing

When inference dominates the workload, the optimal chip is no longer necessarily the one that wins training benchmarks.

That logic is playing out in real hardware decisions right now. DeepSeek is reportedly building its own inference-specific chip, deliberately stepping back from Nvidia and Huawei silicon. The explicit design goal is inference efficiency — not raw training throughput. AWS's Trainium3 reaches approximately 362 MXFP8 PFLOPs at rack scale. Nvidia's NVL72 sits at roughly 360 PFLOPs on the same metric. On paper, for inference workloads, they're tied.

The GPU-as-default assumption is cracking. Cerebras has positioned itself on latency. Specialized inference chips from multiple directions are closing the gap that Nvidia held comfortably for the training era.

This isn't a chip story. It's a signal that inference has its own economics — and those economics reward different architectural choices than training did.

The Real Bottleneck Isn't Supply

Here's the counterintuitive part: GPU shortage and idle GPU supply are being discussed simultaneously, by serious people, as both being real.

That's not contradiction. That's a distribution problem wearing a capacity problem's clothes.

AI demand is outpacing available supply in the right locations, at the right latency, with the right access layer. Meanwhile, millions of deployed GPUs sit underutilized. The shortage is organized routing, not raw silicon.

Projects like RENDER's Dispersed are building directly on this arbitrage — the gap between deployed GPU capacity and actually-utilized GPU capacity. The thesis is that idle compute is the real unlock, not new builds. If you can route inference workloads to existing hardware that's currently doing nothing, you've solved the shortage without manufacturing a single new chip.

This is the same structural insight that made content delivery networks (CDNs) valuable in the early web era. Building more servers wasn't the answer. Routing intelligently to existing ones was.

AntSeed's Position in This Picture

AntSeed is infrastructure for this exact moment. The core bet: inference routing through a decentralized provider network eliminates the gatekeepers that sit between model capability and the people who need it.

When inference is the workload — not training, not fine-tuning, but live, continuous, production serving — the structural question becomes: who controls the routing layer? Centralized aggregators set the terms. They decide pricing, availability, and access. That's fine until it isn't — until a model goes dark, a rate limit hits, or costs spike because a single provider had a quarter to make.

AntSeed routes around that. Providers plug in compute. Buyers get inference. No single point of control. Check live network stats to see the current state of the network — requests handled, providers active, cost per token versus centralized alternatives.

The comparison to OpenRouter and other aggregators isn't abstract. It's architectural. One model centralizes routing control; the other distributes it. When inference is two-thirds of all AI compute and that share is growing, the routing layer is the oil pipeline. The question is who owns it.

What This Means for the Next Two Years

A few things that follow from the inference-first reality:

Model providers will be judged on serving economics, not just benchmark scores. Anthropic's parabolic revenue is a serving story, not a training story.

Chip design will bifurcate further. Training chips and inference chips will increasingly be optimized for different objectives. Buying the best training chip to run inference is like buying a freight truck to do last-mile delivery.

Idle compute becomes a real asset class. The gap between deployed and utilized GPU capacity is large enough to build businesses on — and several projects already are.

Routing infrastructure becomes critical. When inference supply is distributed across data centers, idle consumer hardware, and specialized inference farms, the value lives in the layer that organizes it all.

The oil analogy holds precisely because oil's value wasn't in finding it — it was in refining it, transporting it, and getting it to where it was needed, when it was needed. Inference is the same. The models exist. The compute exists. The value is in the pipe.

Inference overtook training as the dominant AI workload. That's already happened. The hardware industry is catching up to it, the chip design assumptions are shifting because of it, and the revenue numbers from model providers are confirming it.

The bottleneck going forward isn't capacity — it's organized routing of capacity that already exists.