Every System · Tracked

The DGX GB200 NVL72 is NVIDIA's flagship AI compute rack. It contains 72 Blackwell GPUs and 36 Grace CPUs in a single liquid-cooled rack, connected via NVLink 5.0. It delivers up to 720 PFLOPS of FP8 compute and requires 120+ kW of power. It's the system training the largest AI models at Microsoft, Oracle, CoreWeave, and xAI.

Total Cost of Ownership per PFLOPS per year normalizes the cost of different AI systems to a comparable metric. It includes hardware cost (amortized over 3 years) plus power costs (at $0.05/kWh average datacenter rate) divided by the system's FP8 compute output in PFLOPS. Lower is better. This is the metric datacenter operators optimize when choosing between NVIDIA, AMD, Google TPU, or custom silicon.

Modern AI chips consume 700-1200W each. An NVL72 rack with 72 GPUs at 1000W each generates 72 kW of heat from GPUs alone. Air cooling cannot efficiently remove this much heat. Liquid cooling (direct-to-chip or immersion) is 10-100x more effective at heat transfer, enabling higher GPU density per rack, lower PUE (1.1 vs 1.3-1.5 for air), and higher chip performance due to better thermal management.

Google TPU pods and NVIDIA DGX racks are fundamentally different architectures. A TPU v5p pod can contain 8,960 chips in a single interconnected domain via ICI fabric. NVIDIA's NVL72 is a 72-GPU rack-scale system. TPU pods offer massive parallelism for Google's own model architectures but are only available on Google Cloud. DGX systems are available from multiple OEMs for on-premise deployment.

The Cerebras CS-3 uses a single WSE-3 chip, the largest chip ever made, occupying an entire 300mm silicon wafer. Instead of using HBM stacks, it has 44 GB of on-chip SRAM for zero-latency memory access. This eliminates the memory bandwidth bottleneck that limits conventional GPU systems. The tradeoff is higher per-system cost and a different programming model.

Google (TPU), AWS (Trainium), Microsoft (Maia), and Meta (MTIA) are all building custom AI chips to reduce dependency on NVIDIA, optimize for their specific workloads, and lower costs at scale. At hyperscaler volume (hundreds of thousands of chips), even a 10-20% efficiency gain over NVIDIA translates to billions in savings. Custom silicon also provides supply chain diversification.