For greater than a decade, Nvidia’s GPUs have underpinned almost each main advance in trendy AI. That place is now being challenged.
Frontier fashions corresponding to Google’s Gemini 3 and Anthropic’s Claude 4.5 Opus have been educated not on Nvidia {hardware}, however on Google’s newest Tensor Processing Models, the Ironwood-based TPUv7. This indicators {that a} viable different to the GPU-centric AI stack has already arrived — one with actual implications for the economics and structure of frontier-scale coaching.
Nvidia's CUDA (Compute Unified Gadget Structure), the platform that gives entry to the GPU's huge parallel structure, and its surrounding instruments have created what many have dubbed the "CUDA moat"; as soon as a crew has constructed pipelines on CUDA, switching to a different platform is prohibitively costly due to the dependencies on Nvidia’s software program stack. This, mixed with Nvidia's first-mover benefit, helped the corporate obtain a staggering 75% gross margin.
In contrast to GPUs, TPUs have been designed from day one as purpose-built silicon for machine studying. With every era, Google has pushed additional into large-scale AI acceleration, however now, because the {hardware} behind two of essentially the most succesful AI fashions ever educated, TPUv7 indicators a broader technique to problem Nvidia’s dominance.
GPUs and TPUs each speed up machine studying, however they mirror completely different design philosophies: GPUs are general-purpose parallel processors, whereas TPUs are purpose-built methods optimized nearly completely for large-scale matrix multiplication. With TPUv7, Google has pushed that specialization additional by tightly integrating high-speed interconnects instantly into the chip, permitting TPU pods to scale like a single supercomputer and decreasing the price and latency penalties that usually include GPU-based clusters.
TPUs are "designed as a complete 'system' rather than just a chip," Val Bercovici, Chief AI Officer at WEKA, instructed VentureBeat.
Google's business pivot from inside to industry-wide
Traditionally, Google restricted entry to TPUs solely by means of cloud leases on the Google Cloud Platform. In latest months, Google has began providing the {hardware} on to exterior clients, successfully unbundling the chip from the cloud service. Prospects can select between treating compute as an working expense by renting through cloud, or a capital expenditure (buying {hardware} outright), eradicating a significant friction level for giant AI labs that choose to personal their very own {hardware} and successfully bypassing the "cloud rent" premium for the bottom {hardware}.
The centerpiece of Google's shift in technique is a landmark cope with Anthropic, the place the Claude 4.5 Opus creator will obtain entry to as much as 1 million TPUv7 chips — greater than a gigawatt of compute capability. Via Broadcom, Google's longtime bodily design accomplice, roughly 400,000 chips are being bought on to Anthropic. The remaining 600,000 chips are leased by means of conventional Google Cloud contracts. Anthropic's dedication provides billions of {dollars} to Google's backside line and locks certainly one of OpenAI's key opponents into Google's ecosystem.
Eroding the "CUDA moat"
For years, Nvidia’s GPUs have been the clear market chief in AI infrastructure. Along with its highly effective {hardware}, Nvidia's CUDA ecosystem encompasses a huge library of optimized kernels and frameworks. Mixed with broad developer familiarity and an enormous put in base, enterprises steadily grew to become locked into the "CUDA moat," a structural barrier that made it impractically costly to desert a GPU-based infrastructure.
One of many key blockers stopping wider TPU adoption has been ecosystem friction. Up to now, TPUs labored greatest with JAX, Google's personal numerical computing library designed for AI/ML analysis. Nonetheless, mainstream AI growth depends totally on PyTorch, an open-source ML framework that may be tuned for CUDA.
Google is now instantly addressing the hole. TPUv7 helps native PyTorch integration, together with keen execution, full assist for distributed APIs, torch.compile, and customized TPU kernel assist underneath PyTorch’s toolchain. The objective is for PyTorch to run as simply on TPUs because it does on Nvidia GPUs.
Google can be contributing closely to vLLM and SGLang, two well-liked open-source inference frameworks. By optimizing these widely-used instruments for TPU, Google ensures that builders are in a position to change {hardware} with out rewriting their total codebase.
Benefits and drawbacks of TPUs versus GPUs
For enterprises evaluating TPUs and GPUs for large-scale ML workloads, the advantages heart totally on price, efficiency, and scalability. SemiAnalysis not too long ago revealed a deep dive weighing the benefits and drawbacks of the 2 applied sciences, measuring price effectivity, in addition to technical efficiency.
Due to its specialised structure and larger vitality effectivity, TPUv7 presents considerably higher throughput-per-dollar for large-scale coaching and high-volume inference. This enables enterprises to cut back operational prices associated to energy, cooling, and knowledge heart sources. SemiAnalysis estimates that, for Google's inside methods, the whole price of possession (TCO) for an Ironwood-based server is roughly 44% decrease than the TCO for an equal Nvidia GB200 Blackwell server. Even after factoring within the revenue margins for each Google and Broadcom, exterior clients like Anthropic are seeing a ~30% discount in prices in comparison with Nvidia. "When cost is paramount, TPUs make sense for AI projects at massive scale. With TPUs, hyperscalers and AI labs can achieve 30-50% TCO reductions, which could translate to billions in savings," Bercovici stated.
This financial leverage is already reshaping the market. Simply the existence of a viable different allowed OpenAI to barter a ~30% low cost by itself Nvidia {hardware}. OpenAI is without doubt one of the largest purchasers for Nvidia GPUs, nonetheless, earlier this yr, the corporate added Google TPUs through Google Cloud to assist its rising compute necessities. Meta can be reportedly in superior discussions to amass Google TPUs for its knowledge facilities.
At this stage, it would look like Ironwood is the perfect answer for enterprise structure, however there are a selection of trade-offs. Whereas TPUs excel at particular deep studying workloads, they’re far much less versatile than GPUs, which might run all kinds of algorithms, together with non-AI duties. If a brand new AI approach is invented tomorrow, a GPU will run it instantly. This makes GPUs extra appropriate for organizations that run a variety of computational workloads past commonplace deep studying.
Migration from a GPU-centric setting will also be costly and time-consuming, particularly for groups with present CUDA-based pipelines, customized GPU kernels, or that leverage frameworks not but optimized for TPUs.
Bercovici recommends that corporations "opt for GPUs when they need to move fast and time to market matters. GPUs leverage standard infrastructure and the largest developer ecosystem, handle dynamic and complex workloads that TPUs aren't optimized for, and deploy into existing on-premises standards-based data centers without requiring custom power and networking rebuilds."
Moreover, the ubiquity of GPUs means that there’s extra engineering expertise out there. TPUs demand a uncommon skillset. "Leveraging the power of TPUs requires an organization to have engineering depth, which means being able to recruit and retain the rare engineering talent that can write custom kernels and optimize compilers," Bercovici stated.
In apply, Ironwood’s benefits may be realized largely for enterprises with massive, tensor-heavy workloads. Organizations requiring broader {hardware} flexibility, hybrid-cloud methods, or HPC-style versatility could discover GPUs the higher match. In lots of instances, a hybrid method combining the 2 could provide one of the best stability of specialization and adaptability.
The way forward for AI structure
The competitors for AI {hardware} dominance is heating up, but it surely's far too early to foretell a winner — or if there’ll even be a winner in any respect. With Nvidia and Google innovating at such a fast tempo and firms like Amazon becoming a member of the fray, the highest-performing AI methods of the long run may very well be hybrid, integrating each TPUs and GPUs.
"Google Cloud is experiencing accelerating demand for both our custom TPUs and Nvidia GPUs,” a Google spokesperson told VentureBeat. “As a result, we are significantly expanding our Nvidia GPU offerings to meet substantial customer demand. The reality is that the majority of our Google Cloud customers use both GPUs and TPUs. With our wide selection of the latest Nvidia GPUs and seven generations of custom TPUs, we offer customers the flexibility of choice to optimize for their specific needs."
