#gpu-clusters
- 5 min read
HGX to NVL72: What Changes When the Rack Becomes the Server
An 8-GPU HGX node and a 72-GPU NVL72 rack look like the same idea at different sizes. They aren't. Here's what actually changes in the NVLink domain, the network, power, cooling, and how you plan a deployment.
#gpu-clusters#nvlink#nvl72#hgx
- 4 min read
Training vs Inference: Two Different Infra Problems
Same GPUs, very different clusters. Training is a synchronized throughput problem. Inference is a memory-bandwidth and KV-cache problem, which is why prefill/decode disaggregation is taking over. The math, with Llama 3 70B as the example.
#training#inference#gpu-clusters#kv-cache#economics
- 4 min read
Checkpoint Storage That Doesn't Stall Training
Meta's Llama 3 run hit an interruption every ~3 hours on 16K GPUs. At 100K GPUs, checkpoint and restart time decide your goodput. A worked model of checkpoint size, bandwidth, and optimal interval, plus what it costs when you get it wrong.
#storage#training#gpu-clusters#reliability#economics
- 4 min read
Fat Trees, Leaf-Spine, and How Many Switches You Actually Need
A worked example of sizing an 8,192-GPU scale-out fabric: rails, radix, oversubscription, switch counts, and the optics bill nobody budgets for.
#networking#gpu-clusters#fabric#planning#optics
- 4 min read
InfiniBand vs Spectrum-X: Picking a Scale-Out Fabric
Quantum-X800 InfiniBand and Spectrum-X Ethernet both ship 800 Gb/s AI fabrics. Ethernet is winning on share, InfiniBand is having a rebound, and the right answer depends on operations, tenancy, and how much you value SHARP.
#networking#infiniband#spectrum-x#gpu-clusters#ethernet