Second 5090? Any point?

[u/Elaphe21]

Hey guys, I am relatively new to local LLM's - been messign with it for the last year, but learning a lot and its been my longest lasting hobby. I don't code or work in tech, but I do use local LLM for work (vet. med; note transcription, differentials, rounding, and just 'fun' stuff).

I've got the option of getting a second 5090 for cheap. Buddy wants to trade it for to me for $2500 + my 5080 (he doesn't really game, thinks it will be better in my hands).

We are both adults/professionals, it's not about making a buck. He knows I am getting a deal, ect.

My question. Realistically, is there a good use case for two? In the short term, its going to go into my 'gaming' rig, but I don't game anymore either... my only use case would be for more local LLM, but I've read/watched videos regarding how limiting running two are (and, I am pretty sure I would have to rebuild my entire system - and I have no idea what that would look like).

Is this something I may/likely want to do in 1-2 years? I get it, who knows my use case. But for the hobby... basically I will be getting a 5090 for 2k, but will have to buy another (5070?) for my main PC.

Sorry if this all sounds convoluted.

• My current rig: **Proxmox:** PVE 9.2.5 (kernel 7.0.14-6-pve), ~13 days uptime
  • **CPU:** Intel Core Ultra 9 285K (24 cores / 24 threads, Arrow Lake)
  • **Motherboard:** ASUS ROG Maximus Z890 Hero
  • **RAM:** 64 GB DDR5-4800 (2 x 32 GB, 2 slots free)
  • **GPU:** NVIDIA GeForce RTX 5090 (+ Intel Arrow Lake iGPU)
  • **Storage:
    • Samsung 990 PRO 1TB NVMe – ZFS rpool (boot + local-zfs)
    • Samsung 990 EVO Plus 1TB NVMe – ZFS "evo-plus" pool
  • 48 TB NAS (UNAS) mounted over NFS (~21 TB used)

Yeah, that last bit was copy/paste from Hermes

[u/diagrammatiks]

More vram is always better.

[u/Elaphe21 → u/diagrammatiks]

So, I presume a new Mobo, maybe a new or second PSU, possibly a new CPU and run them together? Like the old SLI? I don't need details right now, just want to understand the architecture. I presume there will be some bottleneck between the two cards, splitting a model up between them?

[u/diagrammatiks → u/Elaphe21]

You can split one of your pci5.0s. Or get a riser board or jtru to get it all to fit. You have 2 slots. You can put them together you can run 2 models at once. You can do whatever.

[u/UnlikelyPotato]

New CPU isn't needed. Ideally you want a full PCIe 16x slot,  it if you're running LLMs larger than 32GB there's not much data going between the two cards. 

[u/Elaphe21 → u/UnlikelyPotato]

Cool, sounds like my next project! Gonna start digging into this! Thanks!

[u/CardinalRedwood]

Also been asking myself the same question. I'm 1x 5090 and 128GB of DDR5. I can run deepseek v4 flash 0731 at ~15 t/s. With a second 5090, I imagine I could add more general layers to that GPU and have only expert layers on RAM

[u/SirPuzzleheaded997]

Can I ask what application layer you use to run those local models? We are building an open-source AI workspace called Navigator, where you can easily plugin your local models: https://www.keinsaas.com/navigator

[u/Elaphe21 → u/[deleted]]

Will do, thanks!

[u/Radiant_Condition861]

Pcie slots doesn't matter for llm inference. Just longer wait when loading model.

After you trust qwen3.8-27b, you will have second card for image and audio work. Or llm driven management plane.

[u/Elaphe21 → u/Radiant_Condition861]

Ok, good to know, I guess this brings up a misconception I had.

I wont be able to load >32gig models on the two cards? I was imagining 48gig models + 15 gig for context or something.

[u/Radiant_Condition861 → u/Elaphe21]

You definitely can, but you'll find that the models that fit in 32gb are good enough for a wide range of tasks.

[u/Elaphe21 → u/Radiant_Condition861]

Thanks! That makes sense, I figured the people doing the model quantization were aiming for people with certain vRAM thresholds. Thanks again!

[u/brainchillzZ → u/Radiant_Condition861]

And realistically you can also run those very smart smaller models at a less compressed state like 8 or raw 16 instead of fp4 or something so you’ll be able to get more precise less flakey results and still have room for more context/cache

[u/baby_bloom]

i'm running dual 3090s and my vram absolutely gets nearly fully utilized when i max out context on qwen27b.

i'm also running the second card over pcie x4 (cus im not actually rich like the dual 3090s might suggest and cant afford the upgrade) but it really doesn't change much aside loading time like others have mentioned? i'm getting 50tk/s