Multi-GPU for Local AI in 2026: NVLink vs PCIe and When a Second Card Actually Helps

This article was originally published on among consumer GPUs. The , and every other Ada or Blackwell GeForce card does not support it. If you are running any of those, you are doing multi-GPU over PCIe whether you know it or not.

That is not necessarily a problem. But it does change what you should expect, how you should configure your software, and whether adding a second card is worth it at all. This guide covers all three questions with verified numbers.

NVLink on consumer GPUs: the short and definitive history

NVLink is NVIDIA's proprietary high-bandwidth GPU-to-GPU interconnect. On data center hardware it provides extraordinary bandwidth — 600 GB/s on A100s, 900 GB/s on H100s. On consumer hardware the story is much simpler: NVIDIA supported NVLink on exactly one consumer GPU generation (Ampere, 2020–2021), then removed it entirely.

Here is the full consumer NVLink support table:

GPU	Architecture	NVLink support	Bandwidth
RTX 2080 Ti	Turing	Yes (NVLink 2.0)	100 GB/s
RTX 3090	Ampere	Yes (NVLink 3.0)	112.5 GB/s
/ 4080 / 4090	Ada Lovelace	No	—
with a 14B model that fits comfortably in 24 GB of VRAM, Ollama will automatically detect your second GPU and split layers across both cards. The result, counterintuitively, is slower inference — because every token now requires PCIe data transfers between cards that were not necessary when the model lived on one GPU. Ollama's official documentation confirms this behavior: second GPU accelerates large models that require VRAM pooling; it hurts small models that would otherwise run fully on one card. Scenario 3: High-concurrency serving. If you are running
Ollama, model fits on one card	No	Ollama adds overhead, not speed

The RTX 3090 NVLink setup: what it actually buys you

For home-lab users who specifically want NVLink, this is the only practical path. Two used RTX 3090s connected with an NVLink bridge give you:

• 
  48 GB combined VRAM — enough for Llama 3.3 70B at Q4_K_M with context headroom


• 
  112.5 GB/s GPU-to-GPU bandwidth — ~3.5× the throughput of PCIe 4.0 x8


• 
  50% throughput improvement over running the same two 3090s without NVLink in tensor-parallel configurations

Hardware required:

• Two RTX 3090 cards (NOT 3090 Ti — that card has no NVLink connector)


• One NVIDIA NVLink Bridge 4-slot (ASIN B08S1RYPP6 on Amazon, also available from Newegg). Originally $79 MSRP; as of May 2026, available on Amazon and eBay in the $50–80 range


• A motherboard with two PCIe x16/x8 slots with sufficient slot spacing for the 4-slot bridge

The thermal reality: Two RTX 3090s at full inference load draw approximately 350W each, putting the combined GPU power draw at ~700W. The NVLink bridge sits between the cards, blocking airflow between them. A dual-3090 NVLink rig almost always requires aftermarket solutions — open-air cases, additional case fans directly above the GPU stack, or liquid cooling. The

Multi-GPU over PCIe: dual RTX 4090 and beyond

For the majority of multi-GPU home-lab builds in 2026 — dual RTX 4090, dual RTX 5090, any combination without NVLink — PCIe is the interconnect. Here is what to expect.

Dual RTX 4090 running Llama 3.3 70B Q4: approximately 25–30 tokens/sec generation speed with vLLM tensor parallelism. A single RTX 4090 cannot run this model at all (insufficient VRAM), so the comparison i

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen