News — AI Infrastructure

One A40 48GB runs 27–34B models at 23–29 tokens/sec, fits a 70B at Q4 at 12–13.5 tokens/sec, and serves 1,705 tokens/sec of aggregate output on an 8B model at 100 concurrent requests. The measured numbers, the KV cache math behind the 72B cliff, and the three things an Ampere card cannot do.

An NVIDIA A40 48GB works in a Dell PowerEdge R740xd — but the card is not the purchase. The full bill of materials: GPU enablement kit 330-BBMO (US) / 490-BEIX (EMEA), low-profile heat sinks in both CPU sockets, the high-performance fan kit, air shroud and mylar, a dedicated GPU power cable, and PSUs of 1100W or higher — plus the two mistakes that quietly ruin most self-assembled builds.

NVIDIA's NVLink 6 switch moves 3.6 TB/s to every GPU and 260 TB/s across a Vera Rubin NVL72 rack. Here is what it actually does, what it makes possible, and why it forces a rebuild of memory, power, cooling and cabling around it.

NDR InfiniBand, carried by MMA4Z00-NS400 and MMA4Z00-NS optics, is the scale-out fabric behind NVIDIA's own DGX H100/H200 SuperPOD reference architecture. Here's the real problem it solves, who buys it, and how to size an optics order for it.

The flat-top module goes in the NIC, the finned-top goes in the switch. Here's the rule, the NVIDIA and Mellanox part numbers, and the third variant almost nobody stocks.

DAC, AOC, or AEC — confused which interconnect fits your AI cluster? A clear, practical breakdown by distance, power, and rack position, with compatible cable options.

OSFP and QSFP-DD aren't interchangeable. A practical guide to the differences, which NVIDIA and switch hardware uses each, and how to avoid a mismatched order.

A short, plain-English guide to OSFP direct attach copper: what it is, how far 400G and 800G passive cable really goes, flat top vs finned top, and the five deployment scenarios where it is the obvious right answer.