Fat Trees, Leaf-Spine, and How Many Switches You Actually Need

Every AI cluster plan eventually gets to a whiteboard with boxes and lines and someone asking, "So how many switches is that?" The answer falls out of three numbers: endpoints, switch radix, and oversubscription. Here's the math, worked through end to end, plus the mistakes that only show up once the GPUs are sitting on the dock.

TL;DR

• A non-blocking two-tier fat tree built from radix-k switches tops out at k²/2 endpoints. Three tiers gets you k³/4.
• With 144-port Quantum-X800 switches, two tiers covers 10,368 GPUs, which is the number NVIDIA itself quotes (NVIDIA XDR switch docs). With 64-port 800G Ethernet boxes, it's only 2,048, so radix is everything.
• A non-blocking 8,192-GPU rail-optimized fabric is about 180 switches and ~33,700 800G port-ends. The optics on those ports are often the biggest single line item in the fabric.

Step 1: rails, not racks

The modern AI fabric is rail-optimized. GPU 0 in every server cables to the Rail-0 leaf, GPU 1 to the Rail-1 leaf, and so on. That's the layout NVIDIA's SuperPOD-style reference designs use: same-rail GPUs end up one switch hop apart, and collective libraries like NCCL are built to take advantage of it. Most all-reduce traffic in a ring or tree goes between same-index GPUs, so a lot of it never reaches the spine.

With NVL72-class racks, the scale-up domain (72 GPUs on NVLink) handles tensor and expert parallelism. The scale-out fabric carries data-parallel and pipeline traffic. That split is why fabric sizing became an inter-rack problem.

Step 2: radix sets the ceiling

For a non-blocking fat tree, each leaf uses half its ports going down and half going up.

• Two tiers: max endpoints = k × (k/2) = k²/2
• Three tiers: max endpoints = k³/4

Switch	800G ports (k)	2-tier max	3-tier max
51.2T Ethernet (e.g. SN5600)	64	2,048	65,536
Same box broken out to 128 × 400G	128 (400G)	8,192 @400G	524,288 @400G
Quantum-X800 Q3400	144	10,368	746,496
Spectrum-X Photonics SN6800	512	131,072	huge

This is why the 144-port Quantum-X800 and the 512-port SN6800 matter so much. Every tier you remove takes out a layer of switches, a layer of optics, a hop of latency, and a pile of failure points. A 10k-GPU cluster that fits in two tiers is much cheaper and easier to run than one that needs three.

Step 3: work the example

Setup: 8,192 GPUs, one 800G NIC per GPU, 144-port switches, non-blocking, rail-optimized with 8 rails.

1. Per rail: 8,192 / 8 = 1,024 GPUs.
2. Leaves per rail: each leaf gives 72 ports down, so 1,024 / 72 = 14.2 → 15 leaves (56 spare ports per rail for growth or spares).
3. Total leaves: 15 × 8 = 120.
4. Uplinks: 120 × 72 = 8,640.
5. Spines: 8,640 / 144 = 60.
6. Total switches: 180.

Now count port-ends, since each one needs a transceiver or a cable end:

Segment	Port-ends
GPU NICs	8,192
Leaf downlinks	8,192
Leaf uplinks	8,640
Spine ports	8,640
Total	~33,700

That's about 4.1 port-ends per GPU. With NVIDIA pegging a pluggable DSP transceiver at roughly 30 W per interface (NVIDIA), optics for a fully optical build like this land somewhere around 0.5–1 MW depending on module type. Some short GPU-to-leaf runs can use copper, but at end-of-row distances most can't.

Step 4: should you oversubscribe?

Same cluster at 2:1 at the leaf (96 down / 48 up):

• Leaves per rail: 1,024 / 96 → 11 → 88 leaves
• Uplinks: 88 × 48 = 4,224 → spines: 30
• 118 switches, ~24,800 port-ends. That's about 26% fewer optics.

Is it worth it? For training, usually not. Synchronous jobs run at the speed of the slowest link, and the GPUs are 80–90% of the capex. Saving 26% on the network to idle GPUs during all-reduce is a bad trade. For inference fleets, where most traffic stays inside the scale-up domain or within one node, oversubscription is often fine. Right-size it to the workload.

The mistakes that show up after the GPUs arrive

• Optics lead times. Switches show up, then you wait weeks for 800G transceivers. Order optics with the switches, and include spares (I'd start at ~2–3%).
• Fiber plant. 33k port-ends means more than 16k fiber links plus MPO trunks, patch panels, and cable trays sized for them. Pathways get overlooked all the time.
• Forgetting the other networks. Storage, front-end/north-south, and out-of-band management are separate fabrics with their own switch counts. The back-end math above covers none of them.
• No growth plan. If 8,192 might become 16,384, a two-tier 144-port design won't stretch that far (the ceiling is 10,368). Decide now whether you'll reserve spine ports for a third tier or build a second pod.
• Skipping burn-in. Run NCCL all-reduce and all-to-all across every rail before handing the cluster over. Bad optics and flapping links are much cheaper to find before a 30-day training run.

Bottom line

The switch count is easy once you know the radix. The real cost is the optics and fiber hanging off every port, which is why the industry is chasing higher-radix switches and co-packaged optics. Do the math per rail, count port-ends rather than switches, and don't oversubscribe a training fabric to save a few bucks. That's how you end up cooked.