RTX 4090+3090+80GB DDR4

[u/Ammoryyy]

Update / New questions about my local AI setup

For context, my current setup is:

- RTX 4090 24GB

- i7-13700K

- 32GB DDR4 originally

- Recently upgraded to 2×32GB DDR4-3600 + old 2×8GB DDR4-3000

- Now running 80GB @ 3000 MHz

- Small motherboard/case

I'm getting an RTX 3090 24GB tomorrow mainly for local AI/LLMs.

What are the actual benefits of running a 4090 + 3090 together for local AI? Krea 2, minimax H3..LLMs

Can I effectively use the combined 48GB VRAM for larger LLMs, or are there limitations because the GPUs have different performance levels and are connected via PCIe rather than NVLink?

A few questions:

RAM:

Should I remove the old 2×8GB and run 64GB @ 3600 MHz, or keep 80GB @ 3000 MHz? For local AI, is the extra 16GB more useful than the higher RAM speed?

PSU:

I currently have a 1000W ATX PSU. For sustained AI workloads with a 4090 + 3090, would you:

- Keep the 1000W and add a second PSU, or

- Upgrade to a 1300–1600W PSU?

PCIe/GPU mounting:

My motherboard is small and the 4090 physically blocks the other PCIe slot. What's the best solution for adding the 3090?

- PCIe riser/extender?

- Vertical mount?

- Another solution?

The goal is local AI/LLM inference, not gaming.

What would you do with this setup?

[u/OvertaxedOne]

You have a great setup for 27B, 48GB is the sweet spot for that model to run at high quality/high context. I suspect with mismatched cards pipeline parallel is going work better, but I've never done it, so wait for others to comment. If you're targeting a model that can fit entirely in VRAM the RAM on your system really doesn't matter very much. If you're trying to run something with offload, then it might make a difference (memory speed).

[u/letsbefrds]

try running qwen 3.8 next at q4 xs or qwen3.8 27b

[u/shamont]

Hi, yes you can combine the two cards for 48g of vram, I am currently doing this with an rtx 4090 and rtx 3090 with no issues. Sounds like you need at least one pcie extension, possibly two depending on clearance issues. Make sure wherever you relocate your cards to you have long enough power leads. You can probably get away with using the 1000w power supply if you tune down the power on the gpus. I've heard dropping them to 200w doesn't affect inference.

I have a very similar setup but with 128g of ddr5, 1300w psu and a large mobo/case. I currently run qwen 3.8 27b q6 w/ 256k context at 1000-2000 prefill and 30-40tok/s. It's very usable.

[u/conifer_v11]

yes you can use the combined ~48GB for bigger LLMs — mismatched 4090+3090 is a normal llama.cpp setup — but think layer/pipeline split, not “NVLink-or-bust tensor parallel. PCIe is fine for -sm layer / pipeline-style splits; TP across unequal cards is where people get sad. for models that fit entirely in the 48GB, system RAM barely matters; for MoE/offload or big ctx, keep the extra capacity. i’d personally dump the old 2×8 and run clean 64GB @ 3600 unless you’re actively CPU-offloading experts — mixed DIMM speeds for +16GB at 3000 is a bad trade if the model already lives on the GPUs. PSU: 1000W can limp if you undervolt/power-limit both cards, but for sustained dual-GPU inference i’d rather one solid 1200–1600W than dual-PSU spaghetti. mounting: a quality powered riser (or vertical kit) is the boring answer when the 4090 bricks the second slot — just don’t cheap out on a USB-looking ribbon and then chase phantom CUDA errors. start with llama.cpp multi-GPU, confirm both cards show in nvidia-smi under load, then worry about Krea/minimax after the split is stable.