AI Networking & Fabrics Guide

How GPU servers talk to each other: InfiniBand versus RoCE Ethernet, ConnectX adapter part numbers, PCIe bottlenecks, HDR100 and NDR splits, and the lossless-Ethernet settings that decide tail latency. Pair it with the interconnect matrices for Hopper, Ampere and Volta.

Start here

1InfiniBand or Ethernet?InfiniBand vs. Ethernet for AI Clusters in 2026: Which Fabric Is Right for Your Stack? If you've been following the AI infrastructure space, you've probably heard the debate more than once: InfiniBand or Ethernet? For most of the last decade, InfiniBand was the... Read the guide → 2Cabling the clusterWhat Cable Do I Need? The Complete Guide to GPU Cluster Interconnects in 2026 What Cable Do I Need? The Complete Guide to GPU Cluster Interconnects in 2026 Building a GPU cluster is mostly a hardware procurement problem — until you hit... Read the guide → 3Decode a ConnectX partHow to Read a ConnectX Part Number: MCX4121A-ACAT to MCX75310AAS-NEAT Decoded The letter after the dash is the speed: X 10GbE, A 25GbE, C 100GbE, E EDR/HDR100, H 200G, N 400G NDR. The full ConnectX-4 to ConnectX-7 decoder, the... Read the guide → 4Running RoCELossless Ethernet for RoCE: Why Cable Quality Affects Tail Latency (and What to Buy) Short answer: in a lossless RoCEv2 fabric, tail latency is driven less by average bandwidth and more by how often links have to retransmit or trigger flow control.... Read the guide →

Adapters and fabric switches

MCX75310AAS-NEAT NVIDIA Mellanox ConnectX-7 Single-Port NDR 400Gb/s InfiniBand OSFP Adapter MCX75310AAS-NEAT - Refurbished - NVIDIA Mellanox ConnectX-7 Single-Port NDR 400Gb/s Inf... $1,699.00In stock MCX755106AS-HEAT NVIDIA Mellanox ConnectX-7 VPI Dual-Port 200Gb/s NDR200 InfiniBand & 200GbE QSFP112 Adapter MCX755106AS-HEAT - Refurbished - NVIDIA Mellanox ConnectX-7 VPI Dual-Port 200Gb/s NDR20... $1,649.00In stock MCX713106AS-CEAT NVIDIA ConnectX-7 dual-port 100GbE QSFP112 NIC, New, front view with full-height bracket MCX713106AS-CEAT - New - NVIDIA Mellanox ConnectX-7 EN Dual-Port 100GbE QSFP112 NIC, PC... $1,755.00In stock MCX653105A-HDAT NVIDIA Mellanox ConnectX-6 VPI Single-Port HDR 200Gb/s InfiniBand & 200GbE QSFP56 Adapter MCX653105A-HDAT - Refurbished - NVIDIA Mellanox ConnectX-6 VPI Single-Port HDR 200Gb/s ... $799.00In stock MCX653105A-ECAT NVIDIA Mellanox ConnectX-6 VPI Single-Port HDR100 InfiniBand & 100GbE QSFP56 Adapter MCX653105A-ECAT - Refurbished - NVIDIA Mellanox ConnectX-6 VPI Single-Port HDR100 Infin... $549.00In stock MCX623106AN-CDAT NVIDIA Mellanox ConnectX-6 Dx EN Dual-Port 100GbE QSFP56 NIC, PCIe 4.0 MCX623106AN-CDAT - Refurbished - NVIDIA Mellanox ConnectX-6 Dx EN Dual-Port 100GbE QSFP... $849.95In stock MCX516A-CDAT NVIDIA Mellanox ConnectX-5 Ex EN Dual-Port 100GbE QSFP28 NIC, PCIe 4.0 MCX516A-CDAT - Refurbished - NVIDIA Mellanox ConnectX-5 Ex EN Dual-Port 100GbE QSFP28 N... $499.00In stock MCX556A-ECAT NVIDIA Mellanox ConnectX-5 VPI Dual-Port EDR 100Gb/s InfiniBand & 100GbE QSFP28 Adapter MCX556A-ECAT - Refurbished - NVIDIA Mellanox ConnectX-5 VPI Dual-Port EDR 100Gb/s Infin... $469.00In stock MCX512A-ACAT NVIDIA Mellanox ConnectX-5 EN Dual-Port 10/25GbE SFP28 NIC MCX512A-ACAT - Refurbished - NVIDIA Mellanox ConnectX-5 EN Dual-Port 10/25GbE SFP28 NIC $239.00In stock MSN2700-CS2F NVIDIA Mellanox SN2700 32x 100GbE QSFP28 Onyx switch, refurbished, elevated front view NVIDIA Mellanox MSN2700-CS2F SN2700 32x 100GbE QSFP28 Spectrum Switch — Onyx, P2C Airfl... $2,900.00In stock DCS-7060DX4-32-F Arista 7060X4 32x 400GbE QSFP-DD switch, refurbished, angled front view with rack ears Arista DCS-7060DX4-32-F 7060X4 32x 400GbE QSFP-DD + 2x SFP+ Switch — Front-to-Rear Airf... $12,490.00In stock

Reference tables & compatibility

Every AI Networking & Fabrics article

Buying Guide

Comparison

Troubleshooting

Explainer

Market & Pricing