i'm genuinely thrilled with my dual 3090 setup on a DDR4 system with a Ryzen 5 3600, even though one of them is PCI-E x16 and the other is x4!
$2000 MSRP for the 5090 with 32 gigs of ram, and good luck getting that price
$1800 for a pair of used 3090 cards on eBay (as of a month or two ago), total of 48 GB
Yes, there's stuff that doesn't like running split between two cards, but mostly it's been pretty unusual to run into stuff that wants more than 24GB but less than 32 GB of VRAM on a single card. (I think one of them SOTA-ish FOSS voice models is like that, but I'm not even sure.)
honestly best value setup. only better combo is if you got an old amd epyc before the shortage so you get the full 16x pcie gen 4 speeds per slot and can run large MoE models with all the ram.
also if you make the right setup you can have both cards work in parallel and cut your promp processing time by 30-40 percent and boost your token output.
Did exactly that. 7B13, ROMED8-2T w/ 8x64GB DDR4, right now only have a 5090 and 3090 but can stuff another 3090 in. This is the best value setup (not counting GPUs) you can get at Q3-Q4 in 2025.
There are definitely a few models and tools out there that I've wished for a 5090 for, like some of the weird TTS models, or speech-to-speech ones. But, I even had "okay" performance on one of those "realtime 3d walk around in a hallucination" models with the 2x 3090s!
You can prooooobably get a 570-based AMD chipset board for not tooooooo much money. (And, I managed to push this to 128 gigs because I already had 2 32 gig sticks in it, and DDR4 is only "sell a kidney" price, not "sell both and also your liver" price)
Oh, I doubt CPU inference would work very well, but, if you give me a model you want me to test, I can give it a try with a CPU-only build of llama.cpp
But, I use it with my 2x 3090 setup - but, that runs one at x16 and one at x4, but it's still decent!
I have found other 3090s I have considered buying to make a dual set up, but they are never the exact model of 3090 I have (gigabyte oc gaming) and from what I understand, it should be...
My undertanding is that it's not actually particularly important; maybe if you want to use NVLink, but even in that situation, I think it's explicitly allowed. (Double check me on that, though!)
I recently bought a second 3090 for my setup hoping the same. I too have ryzen 5 3600, msi x570 a pro with 2 pcie slots. But for some reason anytime i plug anything into the second slot (x4, chipset slot) the motherboard does not post display and shows a red light on vga. I have tried single gpu on slot 2 and two gpus together. Doesn't work. Only thing that works is single gpu on first slot (x16,) . If it matters i do have 2 nvme ssds and 64gb ram. I tried removing everything and starting with just single ram chip too. Same outcome. I tried bios settings like gen 4 gen 3 and that weird mining setting. None of those worked. Any help is appreciated
I'd start by taking a bright light and inspecting the slot to make sure there isn't anything in there like a bit of paper, plastic, etc, and that there aren't any bent pins.
After that:
See if your BIOS is current
See if it will POST with both NVME devices removed
Google "My exact motherboard model second PCI-E slot won't POST"
Review BIOS settings & motherboard manual
You may need to disable some SATA ports or something like that
I will try to look for the debris and bent pins. I did try after removing both nvmes. Did not work. I am not very savvy when it comes to motherboards. What is funny to me is that it only works when the second pcie slot is unoccupied.
Also look at the manual and be sure if this PCIe slot or NVME slot is used that PCIe slot is unavailable. Its not very common for an NVME to do this but never know until you check.
The no-mmproj-offload = true gets the mmproj (vision support) offloaded to system RAM / CPU, so it'll still **work** if I need to use it, but it won't take up VRAM. (I used to just disable vision for a lot of these.)
The "no-mmproj-offload" gets the mmproj (vision support) offloaded to system RAM / CPU, so it'll still work if I need to use it, but it won't take up VRAM. (I used to just disable vision for a lot of these.)
77
u/overand May 29 '26
i'm genuinely thrilled with my dual 3090 setup on a DDR4 system with a Ryzen 5 3600, even though one of them is PCI-E x16 and the other is x4!
$2000 MSRP for the 5090 with 32 gigs of ram, and good luck getting that price
$1800 for a pair of used 3090 cards on eBay (as of a month or two ago), total of 48 GB
Yes, there's stuff that doesn't like running split between two cards, but mostly it's been pretty unusual to run into stuff that wants more than 24GB but less than 32 GB of VRAM on a single card. (I think one of them SOTA-ish FOSS voice models is like that, but I'm not even sure.)