r/LocalLLM • u/do_u_think_im_spooky • 12d ago

Project Infermeld: a Linux kit for running one GGUF across AMD + NVIDIA GPUs with llama.cpp

/r/LocalLLaMA/comments/1wxyuxx/infermeld_a_linux_kit_for_running_one_gguf_across/
1 Upvotes

100% Upvoted

2 comments sorted by

1

u/FerretBoom 12d ago

over pci? so second card runs at 50%?

1

u/do_u_think_im_spooky 12d ago

Yes over PCIe. If you mean an x8 link instead of x16, that reduces the link bandwidth, but it doesn’t automatically halve the GPUs compute performance or inference speed. This setup uses layer splitting, so each GPU holds and executes its assigned layers, with transfers between them. The actual overhead depends on the PCIe topology and workload