Repository navigation
Replies: 4 comments
|
I doubt it. I run a RX 6900 XT in a X4 slot and a RX 7900 TX in a 16X slot with a 30/70 tensor split. I haven't seen anything that indicates that the RX 6900 XT is limited by bus speed. All indications are that it is limited by the slower memory bandwidth. In my case two GPUs increases my VRAM and prefill performance but seems to hurt the token generation performance. |
|
Vulkan should let you use both cards together. In #25051 (comment) :
This tested an older version of the PR with CPU-proxy sync. That path was subsequently removed; the current fast path requires matching driver UUIDs/shared semaphores, so AMD + NVIDIA falls back to generic AllReduce. I’d hold off on the motherboard purchase and try layer split on your current setup first. You can already find out whether the extra VRAM is useful for your models. TP does more communication, so x4 could matter there, but I haven’t seen a 5080 + 9070 XT test showing what x8/x8 actually buys you. Would be interested to see your results if you try it. |
|
Possible, yes. llama.cpp can drive both cards at once even across vendors, since each GPU is talked to through its own backend, CUDA for the 5080 and Vulkan for the 9070 XT. The question is which split mode, and the multi-gpu docs in this repo are pretty direct about it. There are two modes that matter here. Layer split (the default) hands each GPU a contiguous slice of layers and runs tokens through them in sequence. Tensor split ( The docs mark tensor mode experimental and only promise good performance for multiple NVIDIA GPUs on the CUDA backend, no guarantees otherwise. It also does several cross GPU reductions per layer, so it is the one mode that is genuinely bottlenecked by interconnect speed. The numbers already in this thread line up with that. On the dual AMD setup in the table above, tensor came in well below layer for generation, and the other report here found the x4 slot was not the limiting factor, the slower card's memory bandwidth was. So for your pairing, realistically
Which makes the motherboard answer fairly simple. The x8/x8 board is not worth buying for this. Layer mode tolerates a slow interconnect by design, and tensor mode's problems here are maturity and mixed backends, not your x4 slot. If you want to test the tensor path anyway, running both cards under Vulkan costs nothing to try before spending money on a board. |
Uh oh!
There was an error while loading. Please reload this page.
Hello, I was lucky enough to win a 5080 from the summer of rtx sweepstakes and was wondering if it would be worth it to upgrade my motherboard from a PCIe 5.0 x16 and PCIe 4.0 x4 configuration to one that supports PCIe 5.0 x8/x8 bifurcation.
My current setup:
What performance uplift/detriment should I expect from tensor compared to pipeline parallelism with different vendors and compute levels?
All reactions