Google Cloud introduced two new variants of its eighth-generation AI chip on Nov. 9, splitting what had been a single processor family into specialized versions aimed at different corners of the AI infrastructure market — its fourth distinct Tensor Processing Unit launch of this year, according to a detailed technical breakdown of the announcement.
The first, called TPU 8tx, is built on the same silicon as the TPU 8t chip Google introduced earlier this year but is tuned for inference rather than training — specifically, for agentic AI workloads that run many inference requests at once. Google replaced the chip's internal networking, swapping a 3D "torus" layout for a denser dragonfly-style topology it calls "boardfly," which the company says cuts the typical distance data has to travel across the chip by more than eight times and roughly halves latency. The standard pod size shrinks accordingly, from 9,600 accelerators down to 1,152. Google also swapped out its own in-house Axion processor for an AMD Epyc chip as the system's host CPU, a change driven by the Epyc's built-in encryption hardware — a nod to customers in regulated industries who want workloads to stay encrypted until the moment they actually reach the TPU.
A chip built to remember more
The second new chip, TPU 8i Max, tackles a different problem: how much an AI system can hold in memory at once. It pairs two of Google's existing TPU 8i chips into a single socket, doubling raw compute and widening the pool of fast on-chip memory available for a model's context window and key-value cache — the working memory AI systems use to track a long conversation or task. Memory capacity per socket rises by about a third compared with the standard 8i, though each individual chip actually carries less of its own high-bandwidth memory than before, a trade-off the architecture is designed to offset by sharing memory more tightly between the two paired chips.
Google has not named specific customers for either chip. Outside analysts expect both to begin shipping on Google Cloud, and potentially to a handful of customers running their own data centers, sometime in 2027 — part of a chip business that, by one count, is nearing 100 customers as it works to pull share away from Nvidia. The launches arrive as Google, Nvidia and a widening field of custom-chip makers race to carve the AI hardware market into niches — training versus inference, raw throughput versus memory capacity, general-purpose versus security-hardened — rather than competing solely on the broadest, fastest general-purpose chip.
Mark Lohmeyer, Google's vice president for AI and computing infrastructure, pointed to a related software layer the company calls TPU Sync, which can pause an idle AI agent's session and shift its memory off the chip to save on computing costs.
This is exactly what TPU Sync is designed to address.
Mark Lohmeyer, Google vice president for AI and computing infrastructure
