Lossless Ethernet for RoCE: Why Cable Quality Affects Tail Latency (and What to Buy)

Short answer: in a lossless RoCEv2 fabric, tail latency is driven less by average bandwidth and more by how often links have to retransmit or trigger flow control. Cable quality is one of the few variables you fully control — a low-BER passive DAC on short runs beats reaching for optics or longer copper "just in case." Here's how to build the cable plant around that.

Why "lossless" Ethernet cares about your cables at all

RDMA over Converged Ethernet (RoCEv2) needs a fabric that doesn't drop packets, which is why it leans on Priority Flow Control (PFC) and ECN marking to signal congestion before buffers overflow. That system works well when links are clean. It works badly when a link has a marginal bit error rate: every corrupted frame either gets retransmitted at the transport layer or, worse, contributes to the conditions that trigger a PFC pause frame upstream. Pause frames stall traffic on a whole priority queue, not just the flow that caused the problem — which is exactly how one flaky cable turns into a tail-latency spike for everyone sharing that link.

Average throughput numbers hide this. A fabric can post great aggregate bandwidth while still delivering terrible p99 latency for GPU-to-GPU collectives, because collectives (allreduce, all-to-all) are only as fast as their slowest participant. This is why tail latency, not average latency, is the number that actually predicts training step time and inference SLA misses on RoCE fabrics.

The cheapest fix: don't run copper past its clean range

Passive DAC cables have effectively zero added latency and the lowest bit error rate of any interconnect option — when used within their rated reach. The mistake we see most often is stretching a passive DAC to its absolute maximum spec'd length to save a hop, which is exactly the wrong place to cut margin on a lossless fabric. Keep copper runs comfortably within spec for the speed you're running, and let AOC or transceivers handle anything longer instead of pushing copper past where it stays clean.

When you need reach: AOC keeps the fabric lossless further out

Once a link has to cross the row or span racks, active optical cables are the more predictable choice than pushing copper past its rated distance. AOCs convert to optics inside the connector itself, so from the switch and NIC's perspective the link still behaves like a direct-attach cable — no separate transceiver and patch panel to introduce additional connector loss and failure points.

Running 400G or 800G past copper's clean range? Our 400G transceivers and full interconnects catalog cover longer optical links at higher speeds.

Where this is headed: Ultra Ethernet and multi-vendor fabrics

The Ultra Ethernet Consortium (UEC) is defining a next-generation Ethernet transport aimed squarely at this problem — better congestion control and multi-path handling for AI/HPC workloads, designed to reduce reliance on PFC pause storms in the first place. Specs and early silicon are still rolling out industry-wide, so treat UEC as the direction the ecosystem is heading rather than something to build around today. What you can act on now is not locking your cable plant to a single vendor's fabric: as RoCE and (eventually) UEC deployments increasingly mix switch and NIC vendors, cables that only identify to one OEM become a real limitation.

Our custom double-programmed DAC and AOC cables are built for exactly this — each end identifies as the correct vendor independently, so one cable links up across mixed-vendor hardware instead of forcing you to stock a different SKU per vendor pairing.

POs and net-terms available for qualified business accounts.