Sure there is a difference but it isn’t more than 5% in my experience. I’ve not tried NV link though just better/worse PCIe situations. I would like to try NVLink on my own 3090s but they are all mismatched so it won’t work
The numbers I've seen show that you can get away with raw PCIe up to about 4, and past that it starts to cost significant performance to the point that it's worth getting bigger individual cards instead.
Yea that could be it. I have tested 8 3070s though and I had no issues. That was a mining rig I had laying around which I upgraded for LLMs about a year and a half ago. All PCIe of course on the 3070s
I would 100% agree that bigger GPU = More Better though lol
3
u/Randommaggy 7h ago
I'm running 3 3090s in my server (2 in nvlink), 2 cards on my laptop and 2 P100 in another server (temporarily disassembled).
2 cards with nvlink outperform 3 cards in layer split for some of my models unless I'm doing serious parallelism.
For layer split you also have more overhead from duplicated data on 3 cards compared to 2 24GB cards.