Welcome back to the OptimusEdge. Yessss !! Last week you did not here from us. As we dove deep into NVIDIA stack and we got lost, finally out with more insight!

Three different NVIDIA technologies all claim to move data fast.

Only one question actually matters: fast between what?

Get that answer right, and the rest of this decision falls into place in about five minutes.
___________________________________________________________
The Edge Upload: Today’s Insights

A new rival to NVLink just got a real spec number. UALink 1.0 can reportedly scale to 1,024 GPUs in a single switching tier nearly double what two tiers of NVSwitch currently reach.

Why it matters: the first credible open-standard challenger to NVLink's scale-up territory, not just Ethernet catching up on the scale-out side.

Photonics is entering the fabric conversation. NVIDIA's new co-packaged optics for both Quantum-X800 InfiniBand and Spectrum-X Ethernet claim 5x better power efficiency than pluggable transceivers.

Why it matters: as GPU counts climb into the hundreds of thousands, the power cost of the network itself becomes a real design constraint, not an afterthought.

Start here, because it resolves most of the confusion by itself.

NVLink connects GPUs inside one server or rack. InfiniBand and Ethernet connect servers to each other.

That's it. That's the whole starting framework. Everything below is detail on top of that one line.

NVLink isn't really a choice you make it's how the GPUs in one system are already wired together.

A single Blackwell GPU supports up to 18 NVLink connections at 100 GB/s each, totaling 1.8 TB/s of bandwidth.

Zoom out to a full GB200 NVL72 rack, and fifth-generation NVLink delivers 130 TB/s of GPU-to-GPU bandwidth across the whole rack.

For comparison, the previous Hopper generation's NVLink ran at 900 GB/s per GPU Blackwell roughly doubled it.

What this bandwidth actually buys: GPUs sharing memory and working on the same problem as if they were one much larger chip, not several small ones passing notes.

Once data has to leave the rack, NVLink's job ends and InfiniBand's begins.

NVIDIA's current Quantum-X800 platform delivers 800 Gb/s per port, with 144 ports on a single switch.

The previous generation, Quantum-2, runs at 400 Gb/s and is still widely deployed.

The number that actually matters most for real workloads isn't bandwidth it's latency.

InfiniBand switches keep port-hop latency below 100–120 nanoseconds, roughly a third to a half of typical Ethernet switch latency.

InfiniBand also supports RDMA remote direct memory access letting GPUs on different machines exchange data without routing through the CPU first.

For a long time, "Ethernet for AI" meant a real compromise. That gap has been closing fast.

NVIDIA's Spectrum-X platform is Ethernet built specifically for AI, combining Spectrum switches with BlueField-3 SuperNICs to deliver lossless RoCE RDMA over Ethernet instead of InfiniBand's native RDMA.

Real numbers: tuned RoCEv2 delivers roughly 5–10 microseconds of latency meaningfully slower than InfiniBand's nanosecond range, but often fast enough. Ethernet's real advantages are elsewhere: broader multi-vendor support and lower cost.

Here's something worth knowing: you rarely choose the fabric by hand, line by line, in your training code.

NCCL NVIDIA's Collective Communications Library does this automatically.

When a distributed training job starts, NCCL probes the system hardware and picks NVLink for GPUs on the same node, then falls back to InfiniBand or RoCE for GPUs on different nodes.

One line in a PyTorch script (nccl as the backend) triggers all of this detection underneath.

The fabric decision genuinely is a hardware and infrastructure choice but once it's made, the software layer just uses it, without you writing separate code paths for each.

Numbers make this concrete faster than any framework. Here's how the decision actually plays out.

Say you're standing up a 128-GPU training run, at 8 GPUs per server.

That's 16 nodes. Below the roughly 32-node threshold from before Ethernet, well-tuned with RoCE, is a genuinely reasonable choice here.

Cost and easier hiring (Ethernet skills are far more common than InfiniBand skills) both favor it at this size.

Now scale the same job to 512 GPUs 64 nodes. You've crossed the threshold.

This is where Boson AI's real 65-node example from earlier stops being a coincidence and starts being the norm: at this scale, InfiniBand's latency advantage compounds across every synchronization step of training, and the gap between "well-tuned Ethernet" and InfiniBand stops being marginal.

The lesson underneath the numbers: this isn't a religious choice between vendors. It's arithmetic that changes as your node count changes.

This is the part most explainers skip entirely how to actually verify, not just plan.

NVIDIA's own nccl-tests repository on GitHub is the standard tool. Clone it, build it, and run a benchmark with debug logging on:

The output tells you the truth directly.

Seeing NCCL INFO Using network IB means InfiniBand is genuinely active.

Seeing NCCL INFO NET/Socket : Using [0]eth0 instead means NCCL silently fell back to a slow socket path a real, common misconfiguration that looks fine until someone checks.

For an H100 or H200 cluster running NVLink plus InfiniBand NDR correctly, expect to see roughly 400–450 GB/s of effective bandwidth in that test.

Meaningfully lower than that, and something in the setup needs attention before a real training run starts discovering this four hours into an expensive job is a genuinely avoidable mistake.

Here's the practical version, stripped of marketing.

Inside one server or rack, you don't choose NVLink is simply how the GPUs are connected. Between machines, the deciding factor is usually cluster size.

A commonly cited threshold: 32 nodes or more typically favors InfiniBand for latency and scaling; smaller or experimental clusters often run perfectly well on well-tuned Ethernet.

A real, named example: Boson AI built a 65-node H100 training cluster on 400G InfiniBand Quantum-2 switches specifically for predictable scaling at that size.

Takeaway: NVLink, InfiniBand, and Ethernet aren't competing for the same job they solve three different distances. NVLink for inside the rack, InfiniBand for serious multi-rack scale, Ethernet for everything smaller or more cost-sensitive. Know which distance your problem actually is before picking a fabric.

That's today's briefing. If your team has debated "InfiniBand or Ethernet" without first asking how many nodes you're actually running, this one's worth forwarding. Past issues are in the archive. See you tomorrow.

NVIDIA Academy Networking Training: NVIDIA's own deep technical training on Quantum InfiniBand networking, linked directly from the Quantum-X800 product page the real starting point if this is genuinely relevant to your infrastructure work.

NVLink vs. InfiniBand Explained: What is the Difference Between NVLink and InfiniBand? bandwidth, latency, and RDMA basics in one place.

Official Quantum-X800 Docs: NVIDIA Quantum-X800 InfiniBand Platform the current-generation InfiniBand switch specs, straight from NVIDIA

Official Spectrum-X Docs: NVIDIA Spectrum-X Ethernet Platform product pages, documentation, and driver downloads

The Decision Framework in Full: InfiniBand vs. Ethernet: Choosing the Right Network Fabric for AI Clusters the 32-node threshold and the Boson AI case study

Verify Your Own Cluster: NVIDIA/nccl-tests (GitHub) the official benchmark tool used in today's verification steps

NCCL Troubleshooting Docs: NCCL Networking Troubleshooting Guide official steps for diagnosing a misbehaving fabric

That’s it for today !☀

Edge AI is levelling up are you? Until next time, stay curious, stay building, and don’t let your machines take over. 🤖😆

Enjoyed today’s issue? Share OptimusEdge AI with your engineering team. Subscribe to OptimusEdge AI

Your Edge AI Explorer,
Sharat Sami (Let’s connect on LinkedIn)