Can You Build a 4-Node DGX Spark Cluster Without a Switch? The Ring That Works and the Breakout Trick That Doesn't

If you own four DGX Spark or GB10 systems and you are trying to avoid buying a switch, you have probably found the same idea circulating on the NVIDIA developer forums: break each 200G QSFP port into 2×100G, give every node four links, and wire a full mesh. It is a clever idea, and on paper the DGX Spark's networking architecture looks like it was built for it. It does not work. NVIDIA does not enable or validate port splitting on the Spark's ConnectX-7, and operators who have tried report only the first lane coming up. The limit is the NIC, not the budget, and no cable choice gets around it.

That is not the end of the switchless question, though. Owners are now running four Sparks with no switch as a ring: four straight cables, each node wired to its two neighbors, held together by a custom NCCL build. It works, it is fast, and it is not NVIDIA-supported. Below: why the breakout mesh fails, how the four-node ring works and how to cable it in a rack, and when a switch is still the better buy.

Updated October 19, 2026: added a section on when to add a switch at three or four nodes (MikroTik CRS504-4XQ-IN vs CRS804-4DDQ) and corrected the CRS812 port count.

The Idea, and Why It Looks Like It Should Work

The proposal shows up on the NVIDIA developer forums roughly like this: a switch adds latency that hurts tensor-parallel workloads, so build a switchless full mesh instead. Each DGX Spark has two QSFP ports. Four nodes in a full mesh need three links per node. Two ports cannot do three links — unless you split them. Break each 200G port into 2×100G and suddenly every node has four links, which is more than enough for a four-way mesh.

What makes this more than wishful thinking is that the DGX Spark's networking really is built out of 100G halves. The GB10 Grace Blackwell SoC cannot provide more than a PCIe Gen5 x4 link per device. To reach 200Gbps, NVIDIA ran the ConnectX-7 in multi-host mode across two separate x4 links. The practical result is that each physical QSFP port carries two 100G MACs, each tied to its own PCIe x4 connection, and Linux presents four Ethernet interfaces rather than two. Users have to bind the right interface pairs or they get roughly 100Gbps instead of the full 190–200Gbps.

So the hardware already contains the 2×100G split the mesh idea depends on. It is reasonable to assume a breakout cable could simply expose it.

Why It Doesn't Work

The split exists on the PCIe side, not at the port. Operators who have actually attempted breakout on DGX Spark hardware report that the port will not come up as independent links — in the words of one, they "were not able to split a QSFP port into four links" and "have only been able to establish a link on the first QSFP lane." A second report on 2×100G describes the same outcome: the cable is recognized, but only one leg becomes active.

NVIDIA's own position on the forums is that this would mean "using an untested splitter cable that could cause other issues," with no validation of the thermal or power profile a splitter presents to the port. It is worth being precise about the mechanism here, because adapter-side splitting is not universally impossible: NVIDIA documents port-splitting configurations for specific ConnectX-7 SuperNIC models, applied through mlxconfig and requiring a power cycle. Those configurations are documented for pluggable adapter cards — not for the soldered ConnectX-7 the DGX Spark ships with, where NVIDIA exposes no split mode and validates none.

This matters because it reframes the whole debate. The objections raised against the mesh proposal on the forums were about cost and bandwidth oversubscription — both arguable, and both beside the point. Even with cheap passive copper breakouts instead of optics, and even if you accept the oversubscription, the topology still cannot be built. And the arithmetic is against it before the NIC is: a four-node full mesh needs three links on every node, and a Spark has two ports. Four cables between four Sparks gets you a ring, not a mesh — every node one hop from two neighbors and two hops from the third. For a long time that was treated as a dead end. As it turns out, a ring is enough for the way tensor-parallel inference actually talks — more on that below.

What NVIDIA Actually Documents

The DGX Spark user guide is unambiguous: the platform "supports up to three DGX Spark systems connected directly through cables, and up to four systems when using a switch." Two QSFP ports per unit, up to 200Gbps each. The two cable part numbers NVIDIA names as approved are the Amphenol NJAAKK-N911 and the Luxshare LMTQF022-SD-R — both QSFP112, both 0.5 m, and both equivalent to the cable Resilient Tec stocks. Because the port runs at 200 Gb/s, the 200G QSFP56 MCP1650-V00AE30 also works in each direct-cabled row below.

The Switchless Four-Node Ring: What Owners Are Actually Running

The ring is the topology the mesh debate wrote off. Each Spark uses both of its QSFP ports, one to each neighbor, with four straight cables in total — no breakout, no split ports. Every node is one hop from two neighbors and two hops from the node opposite.

The reason it works anyway is how tensor-parallel inference communicates. NCCL's ring all-reduce — the collective that dominates tensor-parallel traffic — only ever passes data to the next node in the ring. A physical ring matches that pattern exactly, so the missing "diagonal" links a mesh would add are rarely needed for this workload.

One owner running GLM-5.3 at tensor-parallel size 4 in vLLM across four switchless Sparks posted a full link matrix to r/DGX_Spark in October 2026. Their reported numbers:

• Bandwidth: about 107 Gb/s per logical 100G half on ib_write_bw, uniform across all 16 directed links — roughly 213 Gb/s per neighbor in each direction when one link is tested at a time
• Latency: 1.4–1.9 µs minimum on ib_write_lat, with p99 around 2.7 µs

Those per-half figures line up with the Spark's two-100G-MACs-per-port design described above: each cable carries two ~100G halves, and you only see the full ~200G when both are bound and used. Keep one limit in mind when reading those numbers: both QSFP ports share the same two PCIe Gen5 x4 links to the GB10, so each Spark has roughly 200 Gb/s of total network bandwidth per direction across both ports. The ~213 Gb/s figure is one link tested on its own; with traffic to both neighbors at once, expect closer to ~100 Gb/s per neighbor. That is the same per-node total a switch-connected Spark gets — a ring spreads it across two links rather than losing any.

The catches are real, and they are software, not cabling. Stock NCCL does not know how to route over a four-node Spark ring. The working builds use a custom-compiled NCCL plus a fabric plan that maps which interface faces which neighbor, re-applied at every boot. That owner's own write-up is blunt that it is "bespoke glue, not NVIDIA-supported." Interface mapping is also easy to get wrong: on the second port pair, a hop can land on f0 on one side and f1 on the other, so assuming matching names on both ends breaks the whole matrix. And the ring is a ceiling, not a starting point — a fifth node means rewiring and re-planning everything, where a switch just takes another cable.

Cabling a four-node ring in a rack

If you rack four Sparks in order 1-2-3-4 and cable the ring 1→2→3→4→1, the closing cable from node 4 back to node 1 spans the entire stack — far longer than the other three. A 0.5 m stacking cable often will not reach, and longer passive copper gets stiff and harder to route.

Cable it folded instead: keep the physical order 1-2-3-4, but wire the ring 1→2→4→3→1. Every node still has exactly two neighbors, and no cable spans more than two units, so all four can be the same short length. Stacked tightly, 0.5 m cables reach. If the units sit on separate shelves, measure the longest hop before ordering — the 1 m MCP1650-V001E30 ($97) gives the slack a 0.5 m cable lacks. Your NCCL fabric plan just needs to follow the cabled order, not the rack order.

Ring or switch?

• Choose the ring if you are staying at four nodes, you are comfortable maintaining a custom NCCL build, and you would rather not wait on a switch — owners in fall 2026 report multi-week to multi-month waits on 400G switches. Cabling cost is four cables: $696 in NJAAKK-N911 stacking cables, or $316 in MCP1650-V00AE30.
• Choose a switch if you want the NVIDIA-supported configuration, stock NCCL, any-to-any connectivity, and room to grow past four. Budget roughly $1,650–$1,750 for switch and cabling.
• Do both: some owners run the ring now and move to a switch when it arrives. The ring cables keep their value either way for two- and three-node setups.

When to Add a Switch at Three or Four Nodes: MikroTik CRS504-4XQ-IN vs CRS804-4DDQ

At three nodes you don't need a switch. NVIDIA documents a switchless three-node ring, and every link runs at the full 200 Gb/s port speed. A switch starts to pay off when you go to four nodes and want the NVIDIA-supported setup, when you'd rather run stock NCCL than maintain a custom ring build, or when you want room for a fifth Spark. Most buyers end up choosing between two MikroTik switches.

CRS504-4XQ-IN: the 100G option. Four 100G QSFP28 ports at a $799 MikroTik suggested price. Each Spark takes one port, so four nodes fill it, and each link runs at 100G, half of what a Spark port can do. Owners on the NVIDIA developer forums report that the Spark's ConnectX-7 links at 100G on a QSFP28 cable (set 100G on both ends if auto-negotiation doesn't). NVIDIA's guide asks for cables rated at least 200 Gb/s, so you're choosing that trade-off. In r/DGX_Spark, owners running four Sparks on a CRS504 say inference held up well. One reported roughly a 10% prefill drop versus direct cabling and no change in token generation. That's one owner's report, not a controlled test. Owners also say the CRS504 is easier to find in stock than the 400G models.

CRS804-4DDQ: the 200G option. Four 400G QSFP56-DD ports at a $1,295 MikroTik suggested price. MikroTik supports splitting each port into two 200G links, so two breakout cables put four Sparks on the switch at full 200G and leave two ports free. Four breakouts reach eight nodes. Set every port to 200G by hand. Owners in fall 2026 report long waits for both the CRS804 and the CRS812.

What to buy, in stock at Resilient Tec: for the CRS504, the MCP1600-C001 100G QSFP28 DAC, 1 m ($45 as of October 19, 2026) or the 3 m MCP1600-C003 ($92 as of October 19, 2026). For the CRS804, the MCP7H60-W001 breakout, 1 m ($149 as of October 19, 2026) or the 2 m MCP7H60-W002 ($179.45 as of October 19, 2026). For a ring, the NJAAKK-N911 ($174 as of November 6, 2026) or the MCP1650-V00AE30 ($79 as of November 3, 2026). All three MikroTik switches in this post use Marvell switch chips, so the RoCE tuning warning below applies to each one.

The Supported Four-Node Route: Add a Switch

If you want the configuration NVIDIA documents, the build is straightforward and cheaper than most people expect. There are two ways to cable it, and which one is right depends mostly on how many 400G ports your switch has.

A port per node: use straight cables. The MikroTik CRS804-4DDQ is the common choice in this class — four 400G QSFP56-DD ports at a $1,295 list price. Four nodes, four ports, so run one 200G DAC from each Spark to its own switch port and skip breakout entirely. QSFP-DD cages are backward compatible with QSFP56 modules, so a standard Ethernet-coded 200G QSFP56 passive DAC plugs straight in: the MCP1650-V in 1 m ($97), 1.5 m (Currently out of stock — request a quote), 2 m ($115) or 3 m ($145). Set the port speed to 200G manually rather than trusting auto-negotiation. Four 2 m cables at $115 each come to $460.

Fewer ports than nodes: break out. The MCP7H60-W002, a 400G QSFP-DD to 2×200G QSFP56 passive breakout DAC at $179.45, turns one switch port into two Spark links. Two of them cover four nodes off two ports. On the MikroTik CRS812-8DS-2DQ-2DDQ, which has two 400G QSFP-DD ports plus two 200G QSFP56 ports, you can put all four Sparks on the two QSFP-DD ports with two breakouts, or use one breakout plus a straight 200G cable in each QSFP56 port. On a four-port switch it still has a use: two breakout cables leave two ports free for growth toward eight nodes. It also comes in 1 m ($149), 1.5 m ($169) and 3 m ($199). Pick the shortest length that reaches without strain, because shorter passive copper is easier to route and cheaper.

On cost, breakout comes out ahead: at 2 m, two breakout cables are $358.90 against $460 for four straight MCP1650-V002E26 cables, and the breakout leaves two switch ports free. At 1 m the gap is similar — $298 for two breakouts against $388 for four straight cables. Straight cables still make sense when your switch has a port per node and you want each Spark on its own independent cable, or when the Sparks sit at different distances from the switch. What does not change either way is the direction: the breakout happens on the switch side, where breakout is a supported, documented function. Each Spark still receives one whole 200G QSFP link into an unsplit port. That is the distinction the switchless mesh proposal gets backwards.

The switch-side breakout is not theoretical. A four-node GB10 build documented on the NVIDIA developer forums ran exactly this configuration — a CRS812 fed by QSFP-DD to 2×200G QSFP56 breakout DACs — and measured roughly 196–198 Gb/s per node pair on iperf3 and about 23.8 GB/s bus bandwidth on an NCCL all-reduce, once MTU 9000 was set and both logical halves of each port were driven in parallel.

Budget roughly $1,650 all in for switch and breakout cabling, or about $1,750 with four straight cables — against $316–$696 in cables for a switchless ring, if you are prepared to run and maintain unsupported NCCL software.

One warning worth having before you buy: getting the links up is not the same as getting the throughput. Four-node GB10 clusters on Marvell-based switches have been reported hitting a hard ~10Gbps ceiling on clean 200G links, traced to bridge hardware offload failing, PFC and trust settings blocking RoCE, and duplicate fabric IPs across rails. Plan on MTU 9000, one subnet per rail, and unique IPs per interface, and budget an afternoon for tuning rather than assuming it works out of the box.

What About the 200G to 2×100G Breakout Cable?

Part numbers like the MCP7H50-H001R30 (200G QSFP56 to 2×100G QSFP56) are real, in stock, and genuinely useful — on switch ports and on adapter cards where splitting is documented and configurable. They are not a workaround for the DGX Spark. Buying one to attempt a switchless four-node mesh will get you one active leg and a dead one. If you want four nodes without a switch, build the ring with straight cables. Either way, you want a whole-port cable at the Spark end, every time.

Quick Checklist

• Two or three GB10 systems: direct cabling, no switch, one QSFP112 stacking cable or MCP1650-V00AE30 per link
• Three units form a ring using both QSFP ports on each node
• Four units, supported: a 400G switch — one MCP1650-V 200G DAC per Spark (1–3 m) if the switch has a 400G port per node, or one QSFP-DD to 2×200G breakout cable per pair of Sparks if it does not
• Four units, switchless: a ring of four straight cables plus a custom NCCL topology — works, not NVIDIA-supported, no room to grow
• Racking a four-node ring: cable it folded (1→2→4→3→1) so no cable spans more than two units
• Never split a Spark's QSFP port — breakout belongs on the switch side only
• Set switch port speed to 200G manually; do not rely on auto-negotiation
• MTU 9000, one subnet per rail, unique IP per fabric interface, and verify hardware offload is active before benchmarking

FAQ

Can you build a 4-node DGX Spark cluster without a switch?
Not as a full mesh, and not with breakout cables — a mesh needs three links per node, each DGX Spark has two QSFP ports, and splitting a port only brings up the first lane. A four-node ring does work: four straight cables, each Spark wired to its two neighbors. Owners are running tensor-parallel inference across four switchless Sparks this way using a custom NCCL build. It is not NVIDIA-supported, the opposite node is two hops away, and it cannot grow past four. NVIDIA documents direct cabling for up to three systems and a switch for four, which remains the supported route.

How many DGX Sparks can I connect directly with DAC cables?
NVIDIA documents three. Two units take one cable point to point; three units form a ring using three cables, with each unit using both of its QSFP ports. Four units can also run as a ring with four cables, but only with custom NCCL software outside NVIDIA support. The NJAAKK-N911 QSFP112 400G-rated stacking cable is $174 at Resilient Tec, so a three-node ring is $475 in cabling with the ring kit and a four-node ring is $696; with the $79 MCP1650-V00AE30 they are $237 and $316 (prices checked November 6, 2026).

How do I cable a four-node DGX Spark ring in a rack?
Fold it. Keep the units in physical order 1-2-3-4 but wire the ring 1→2→4→3→1. Cabled in straight order, the cable closing the ring from node 4 back to node 1 has to span the whole stack. Folded, no cable spans more than two units, so all four can be the same short length — 0.5 m cables reach when the units are stacked tightly, and 1 m cables cover units on separate shelves.

Is a switchless ring faster or slower than a switch?
There is no published head-to-head on DGX Spark yet. One owner's switchless ring measured about 213 Gb/s per link when each link was tested on its own, and 1.4–1.9 µs minimum RDMA latency, while switch builds report 196–198 Gb/s per node pair. Either way, each Spark is capped at roughly 200 Gb/s total per direction, because both QSFP ports share two PCIe Gen5 x4 links — a ring splits that budget between two neighbors, a switch sends it all through one port. The bigger difference is operational: a switch runs on stock, supported NCCL and gives every node a one-hop path to every other; the ring needs custom software and puts the opposite node two hops away.

Do I need a breakout cable for a four-node DGX Spark cluster?
Only if your switch has fewer 400G ports than you have nodes — but it is also the cheaper option. On a four-port switch like the MikroTik CRS804-4DDQ you can run one 200G DAC per Spark and skip breakout entirely. On a switch with only two 200G-capable ports, two breakout cables are the only way to land four Sparks. At 2 m, two breakout cables cost $358.90 against $460 for four straight MCP1650-V002E26 DACs, and leave two switch ports free. A switchless four-node ring uses no breakout at all.

Is a 100G switch like the MikroTik CRS504-4XQ-IN enough for four DGX Sparks?
For many inference setups, owners say yes. The CRS504-4XQ-IN has four 100G QSFP28 ports, one per Spark, so each node links at 100G instead of 200G. Owners running four Sparks on one report small slowdowns compared with direct 200G cabling, mostly in prompt processing. NVIDIA's guide calls for cables rated at least 200 Gb/s, so if you want the full 200G per node or plan to grow past four nodes, the four-port 400G CRS804-4DDQ with two QSFP-DD to 2×200G breakout cables is the better fit.

Why does the DGX Spark show four network interfaces for two ports?
The GB10 SoC cannot provide more than a PCIe Gen5 x4 link per device, so NVIDIA runs the ConnectX-7 in multi-host mode across two x4 links per port. Each physical QSFP port carries two 100G MACs, which Linux presents as separate interfaces. They aggregate to 200Gbps, but only if you bind the correct pairs — mismatched pairing caps you near 100Gbps.

Which cable connects a DGX Spark to a 400G switch?
Either a straight Ethernet-coded 200G QSFP56 passive DAC — one per Spark, since QSFP-DD switch cages accept QSFP56 modules — or a 400G QSFP-DD to 2×200G QSFP56 passive breakout DAC, where one switch port serves two Sparks. Resilient Tec sells the MCP7H60-W002 breakout for $179.45 and the 2 m MCP1650-V002E26 200G DAC for $115 (also in 1 m, 1.5 m and 3 m), all new with a lifetime warranty and free US shipping.

Does a switch add enough latency to hurt tensor-parallel workloads?
A single switch hop adds well under a microsecond on a 400G cut-through switch, which is small next to collective operation times on GB10-class nodes. It is a real cost but rarely the limiting factor — misconfigured RoCE, PFC and MTU settings cost far more throughput in practice than the hop does.

Ready to Buy?

Tested and shipping from Knoxville, Tennessee. New, lifetime warranty, free standard shipping on all US orders, same-day shipping on orders placed before 5:00 pm ET. Prices below checked October 26, 2026.

• NJAAKK-N911 — QSFP112 400G-rated stacking cable, 0.5 m — $174, the two- and three-node link, and four of them for a switchless four-node ring
• LMTQF022-SD-R equivalent — QSFP112 400G-rated stacking cable, 0.5 m — $174, same cable if Luxshare's part number is what you were sent
• MCP1650-V00AE30 — 200G QSFP56 DAC, 0.5 m — $79, the lower-cost link for two, three or four nodes in a ring
• MCP1650-V — 200G QSFP56 DAC, switch runs — one per Spark when the switch has a port per node: 1 m $97 · 1.5 m (Currently out of stock — request a quote) · 2 m $115 · 3 m $145
• MCP7H60-W002 — 400G QSFP-DD to 2×200G QSFP56 breakout, 2 m — $179.45, the switch-side cable when ports are scarce
• MCP7H60-W001 — 400G QSFP-DD to 2×200G QSFP56 breakout, 1 m — $149, when the switch sits directly above or beside the Sparks
• MCP7H60-W01AR30 — 400G QSFP-DD to 2×200G QSFP56 breakout, 1.5 m — $169, when 1 m is just short of the switch but you still want the flexible 30 AWG wire
• MCP7H60-W003 — 400G QSFP-DD to 2×200G QSFP56 breakout, 3 m — $199, when the switch is a few rack units away

Going to four nodes, or mixing GB10 vendors? Send us your node count and switch model and we will confirm exactly what to buy before you order.

You Might Also Need

• Why Connect Two (or More) NVIDIA DGX Sparks? — hardware limits, topologies and what scaling actually buys you
• Does the DGX Spark cable work with every GB10 system? — ASUS, Dell, HP, MSI, Lenovo, Gigabyte and Acer
• Can you cluster different GB10 systems? — mixing DGX Spark, ASUS, Dell, HP and other GB10 brands in one cluster, including a four-node ring
• DGX Spark ports explained — why USB-C isn't your clustering cable
• DGX Spark interconnect reference — approved part numbers, real-world throughput, and what to buy at two, three and four nodes
• Lossless Ethernet for RoCE — what to get right once the switch is in the path

Sources

• NVIDIA DGX Spark User Guide — ConnectX-7 Networking and clustering
• NVIDIA Developer Forums — 4-node DGX Spark cluster without a switch
• NVIDIA Developer Forums — Mellanox QSFP breakout
• NVIDIA Developer Forums — ConnectX-7 200GbE via MikroTik CRS812 and QSFP-DD to 2×200G breakout
• r/DGX_Spark — "I got my fourth spark" (switchless four-node ring, link matrix and setup notes)
• NVIDIA ConnectX-7 Adapter Cards User Manual — Port Splitting Configurations
• NVIDIA MCP7H60-W0xxRyy 400GbE to 2×200GbE DAC splitter cable — specifications
• NVIDIA MCP1650-V0xxEyy 200GbE QSFP56 DAC — ordering information
• MikroTik CRS804-4DDQ product page
• MikroTik CRS504-4XQ-IN product page
• MikroTik CRS812-8DS-2DQ-2DDQ product page
• NVIDIA Developer Forums — GB10 QSFP56 port speed, connecting multiple GB10 with the MikroTik CRS804 DDQ
• r/DGX_Spark — "Two vs Three DGX Sparks: is the third one worth it?" (CRS504 at 100G vs CRS804 discussion)