The NVIDIA DGX Spark is already a remarkable machine on its own — a desktop-sized AI supercomputer packing up to 1 petaFLOP of FP4 AI performance, 128 GB of unified memory, and a full NVIDIA software stack. But there are very real reasons why researchers, developers, and enterprises are looking to link two or more of them together. This post breaks down the why, the hardware limitations, and exactly how far you can scale a multi-Spark cluster.
Why Would You Connect Two DGX Sparks?
The DGX Spark is purpose-built to handle large AI workloads at your desk — fine-tuning models up to 70 billion parameters, running inference on models up to 200 billion parameters, and powering autonomous AI agent pipelines. But some workloads quickly push against the limits of a single node, and that is precisely where linking two units together becomes compelling.
1. Running Larger Models
A single DGX Spark has 128 GB of coherent unified system memory. That is enough to run inference on models up to roughly 200 billion parameters and fine-tune up to 70 billion parameters. Once you connect a second unit, you effectively double the available memory pool, unlocking support for models in the 400B-parameter range that simply cannot fit on one node.
2. Faster Fine-Tuning and Training
Fine-tuning workloads can be parallelized across nodes with near-linear performance scaling. NVIDIA's own benchmarks show that going from one to two DGX Spark nodes for fine-tuning a Nanochat model doubles total token throughput from approximately 18,400 tokens/second to 35,900 tokens/second. For teams iterating frequently on custom models, this halving of fine-tuning time is a significant productivity gain.
3. Faster Inference via Tensor Parallelism
For latency-sensitive agentic workloads, spreading inference across two nodes using tensor parallelism can meaningfully reduce time-to-first-token (TTFT). Running Llama 3.3 70B on two nodes drops TTFT from around 33,415 ms to 21,384 ms — a roughly 36% improvement — and cuts per-token generation time approximately in half.
4. Running More Agents in Parallel
Autonomous AI agent frameworks — such as those built on NVIDIA’s NeMo Agent toolkit — often require multiple simultaneous subagents, each managing long context windows. A single DGX Spark can parallelize several subagents across its GPU, but distributing agent workloads across two nodes increases concurrency capacity and total throughput for multi-agent pipelines.
5. Prototyping at Scale Before Cloud Deployment
A two-Spark setup gives developers a local multi-node environment that mirrors cloud-scale distributed inference and training workflows. You can develop, test, and validate multi-node workloads on-premises — with full data privacy — before deploying to larger cloud infrastructure.
Hardware Limitations: What You Need to Know
The DGX Spark's interconnect capability is built around its NVIDIA ConnectX-7 NIC, which runs at 200 Gbps. This is the single most important hardware constraint to understand when planning a multi-Spark setup.
The ConnectX-7 NIC: Your Bandwidth Budget
Each DGX Spark ships with one ConnectX-7 NIC operating at 200 Gbps. This port supports RDMA over Converged Ethernet (RoCE), which is the protocol used to enable low-latency, GPU-direct communication between nodes. In a two-node direct cable connection, you are working with a single 200 Gbps link between the two machines.
By comparison, a server-class DGX H100 system uses NVLink Switch connections for terabytes per second of inter-GPU bandwidth. The DGX Spark's 200 Gbps Ethernet interconnect is capable for many workloads, but it is a real ceiling for communication-intensive tasks like large-model tensor parallelism.
Memory Bandwidth Per Node
Each DGX Spark delivers 273 GB/s of memory bandwidth from its 128 GB LPDDR5x unified memory. This is substantially lower than the HBM3e memory found in data center Blackwell GPUs, which is why intra-node memory-bound workloads are more constrained than on cloud-scale hardware. When fine-tuning or running inference across two Spark nodes, the interconnect bandwidth between the two machines becomes a secondary bottleneck if the model requires frequent synchronization.
Power Per Unit
Each DGX Spark draws up to 240 watts (with the GB10 chip at a 140 W TDP). A two-unit cluster draws around 480 watts total — a factor to consider for home office or lab deployment, though well within reach of standard power infrastructure.
Scaling Is Not Always Linear
Inference scaling across nodes is near-linear only when inter-node communication is minimal — for example, when running many independent simulations or data-parallel workloads. For communication-intensive tasks like large tensor-parallel inference, scaling is sub-linear because the 200 Gbps NIC becomes a bottleneck relative to the volume of data that must be synchronized between GPUs.
Can You Link More Than Two? Yes — Up to Four
NVIDIA now officially supports scaling DGX Spark up to four nodes. As of early 2026, the platform supports multiple execution topologies, each suited to different workload profiles:
Two-Node Direct Cable Connection
The simplest setup: connect two DGX Sparks back-to-back using a direct QSFP112 400G-rated DAC cable between their ConnectX-7 QSFP ports (each port runs at 200 Gb/s). The two cables NVIDIA approves for this are the NJAAKK-N911 and the Luxshare LMTQF022-SD-R, both 0.5m passive direct-attach copper. No switch is required. This topology is ideal for balanced scaling on larger models (up to ~400B parameters), faster fine-tuning, and general-purpose distributed inference. Setup is straightforward and latency is excellent due to the point-to-point connection.
Three-Node Ring Topology
Three DGX Sparks can be connected in a ring, where each node connects to the next in a loop. This topology is particularly well-suited for fine-tuning larger models or running smaller training jobs. Ring configurations distribute collective communication — like all-reduce operations used in distributed training — efficiently across the three nodes. NVIDIA provides a dedicated playbook for this setup.
Four-Node Cluster with a 200 GbE Switch
Connecting four DGX Sparks requires a 200 GbE network switch. All four nodes connect through the switch, which enables full any-to-any communication between all nodes. This is the highest-performance multi-Spark configuration currently supported, enabling local inference on state-of-the-art models up to 700 billion parameters and communication-intensive workloads. Four-node benchmarks show near-linear fine-tuning scaling — for example, Isaac Lab reinforcement learning FPS scales from 630 on one node to 2,520 on four nodes, and Nanochat fine-tuning throughput reaches ~74,600 tokens/second across four nodes versus ~18,400 on a single node.
Which Topology Is Right for You?
Here is a quick summary of how the four supported configurations compare:
| Configuration | Hardware Needed | Best For | Max Model Size |
|---|---|---|---|
| 1 DGX Spark | — | Large context inference, fine-tuning up to 70B, local agent development | ~200B parameters |
| 2 DGX Sparks (direct) | 200 GbE cable | Faster fine-tuning, larger models, low-latency inference | ~400B parameters |
| 3 DGX Sparks (ring) | 200 GbE cables | Fine-tuning larger models, small training jobs | ~500B+ parameters |
| 4 DGX Sparks (switch) | 200 GbE switch + cables | State-of-the-art model inference, communication-intensive workloads | ~700B parameters |
The Bottom Line
Connecting two DGX Sparks together makes compelling sense the moment your models or workloads outgrow a single node's 128 GB of memory or you need to cut fine-tuning times roughly in half. The hardware foundation — the ConnectX-7 NIC at 200 Gbps per node — is purpose-built for this, and the two-unit direct-cable setup — using an approved QSFP112 400G-rated DAC cable — requires no additional switching hardware at all.
Beyond two units, NVIDIA's support now extends to ring topologies (three nodes) and full switch-based clusters (four nodes), with the latter enabling 700B-parameter inference on your desktop. The main hardware constraints to keep in mind are the 200 Gbps per-node NIC bandwidth, the 128 GB per-node memory ceiling, and the sub-linear scaling on communication-heavy workloads.
For most AI development teams, a two-Spark setup hits a compelling sweet spot: no switch required, near-linear fine-tuning scaling, and access to 400B-parameter models — all on your desk, with full data privacy.
Interested in setting up a multi-Spark cluster? NVIDIA offers official playbooks for connecting multiple DGX Sparks, including step-by-step guides for ring topologies and switch-based four-node clusters, available at build.nvidia.com/spark.