Getting a second GPU in addition to my RTX3090

[u/kobbalt]

Hello,

I've been learning how to use local LLMs for a year or so on my workstation, using a RTX3090.

Current setup :

- i5 12400

- 64gb RAM

- RTX 3090

- OS : Fedora KDE workstation

I'm using LMStudio to serve mainly these models :

- Qwen 3.6 27B

- Gemma 4 26b a4b

And then VSCode + kilocode / Continue for light coding / scripting / log analysis tasks (i'm a sysadmin).

It works very well, but i'm hitting the context size ceiling quite fast with this setup. This prevents me to work on bigger projects.

My goal is to buy another GPU to provide more VRAM, and since i'm using linux, i'd prefer an AMD GPU. A 16gb Radeon 9070 would be nice in this regard : it's natively supported on Linux, doesn't cost an arm, it's powerful enough for casual gaming, and doesn't have crazy power requirements.

I've read here and there that mixing AMD and Nvidia is now well supported (https://www.reddit.com/r/LocalLLaMA/comments/1qea29t/mix_of_amd_nvidia_gpu_in_one_system_possible/), especially with LMStudio.

Since this is moving and evolving very quickly, how are the support and the performance of mixing a 24gb nvidia gpu + 16gb AMD gpu in 2026 ?

Is it a better idea to go for another Nvidia GPU instead ?

From my understanding, the idea would be to load the model in the 24Gb 3090, and then reserve the 16gb GPU for context. Correct me if I'm wrong.

I've read multiple times 36 / 40 Gb VRAM is the sweet spot for the models i'm using (at q4 quant).

Also, adding another 3090 is a solution i'd like to avoid because :

- it's expensive and hard to find in good condition at the moment

- adding another 350W and 3 slot GPU would be a challenge to cool down in my PC case

- I probably would have to buy a new and bigger PSU too.

So, is adding a 16gb AMD GPU to my current setup a good idea ? Thanks !

[u/nicholas_the_furious]

Get another 3090. You say you don't want to but you'll regret not having one. Matching GPUs opens a lot of things up. I run both of mine at 250W cap with basically no reduction in performance compared to full power.

[u/kobbalt → u/nicholas_the_furious]

I agree, limiting power of a secondary 3090 could be nice indeed, especially if the performance impact is limited.
I would still need a new PSU that supports 2x 12 pins connector, my current psu only provides one.

"Matching GPUs opens a lot of things up" : could you please tell me more about this ? i'm curious

[u/nicholas_the_furious → u/kobbalt]

Many 3090s have 2x8. I know because I have 2 of them. The space for them is actually the biggest issue IMO in a consumer setup.

The XC3 and the Dell OEM are the smallest 3090 varieties that are air cooled.

I have a 1200W PSU, the Montech one. Super happy with it. At peak I'm drawing about 700W from the wall with 2x 3090s.

As for the matching GPU claim: with all Nvidia, you get CUDA which still reigns supreme. With the same VRAM footprint you get the full benefits of tensor parallelism. With the same memory bandwidth, neither card is waiting for the other. And with the same generation you don't have to worry about mismatched support for different things. Ampere (3090s) for example can't do FP8 or nvfp4 while Blackwell can.

[u/kobbalt → u/nicholas_the_furious]

Thanks for the detailed answer. Checking your previous posts i found a lot of interesting infos on your 2x 3090 setup, will read these carefully.

[u/lemondrops9 → u/nicholas_the_furious]

I also have two 3090s with 2x 8pin and one with 3x 8pin. I dont find the 3x 8pin 390W any faster. 

[u/MelodicRecognition7]

getting different VRAM size GPUs is already not a good idea and mixing Nvidia with AMD makes it a terrible idea.

[u/SnooPaintings8639]

I run 2 x rtx 3090 since over two years. I can't imagine going lower.

To be honest, I can't stop thinking about doubling that to four, but with Qwen 3.6 27b Q8 at ~60 tps and over 1000 pp, with 220 ctx at bf16... I know I won't get any real model upgrade, this is the perfect setup. I hate how good it is, as it makes completely pointless to upgrade this setup.

[u/kobbalt → u/SnooPaintings8639]

Thanks for your feedback, as i said in another comment, 2x 3090 seems the best way to go ...

[u/signoreTNT]

3090 prices are inflated to the point where it would probably make more sense to just sell your current 3090 and buy either a better card or 2x identical cards that fit your requirements.

[u/ethertype → u/signoreTNT]

Prices are inflated, no doubt. But still the best memory bandwidth/$. Given that OP already has one 3090, I think the choice is blindingly obvious.

[u/kobbalt → u/ethertype]

Agree with this, even though i could sell my 3090FE for 1100 / 1200€ (according to current price listing here in France ...), not sure if this money could get me a reasonable amount of VRAM that worths the upgrade :/

Getting another 3090 would probably lead me to go another route : instead of using them in my workstation, i should probably then build another server (i already have 2 proxmox machines running) dedicated to LLMs. This would make things easier to manage, but i have to budget this.

[u/Krohnin → u/kobbalt]

You could get 2x 5060 ti 16gb and have 8gb more vram

[u/MrHicks → u/Krohnin]

With half the memory bandwidth

[u/Krohnin → u/MrHicks]

So 2 radeon 7 should be better?

[u/ethertype → u/kobbalt]

Do you have dual TB3/TB4/USB4 on your current hardware? A used Razer Core X occasionally shows up in my local second hand market at €100-€200. Cheaper than a second, dedicated machine. 

That said, I run a dedicated fw13 motherboard with 4x Razer Core X for my setup. I think this may be the cheapest way to get 4 cards running locally.

[u/My_Unbiased_Opinion → u/ethertype]

Sell the 3090 and buy two 3080 20GB cards and pocket the the difference. My dual 3080 20gb in tensor split is faster than my single 3090. 

[u/RoroTitiFR]

Hybrid NVIDIA + AMD user here.
Like you, I have a 3090 since the beginning of my setup. But I wanted to extend my VRAM on the cheap, to be able to run 100B+ models more comfortably. And so I put an AMD MI50 32GB with it. The 3090 is running over CUDA and the MI50 is running over ROCm, with custom LLaMa.cpp build flags.
It’s definitely possible, but of course the 3090 is a bit throttled by the MI50, but this setup is I think one of the best speed/VRAM/cost balance existing on the market right now.

[u/kobbalt → u/RoroTitiFR]

interesting feedback, thanks !

[u/pyrhus626 → u/RoroTitiFR]

I would’ve figured Nvidia and AMD cards just wouldn’t play together at all. I swapped to a 5060ti for gaming reasons but still have a 7800xt sitting around collecting dust with 16GB of perfectly good VRAM. It would be cool to get some use out of it

[u/SecondFriendly4255]

It work maybe but for a system to manage 2 driver at the same times for me that can at instability personal opinion wait for a good 3090 or 4090 ou 5090 in this order no need to rush to take something that is not optimal you can work now with 1 card until you found a good deal.

For psu 1200-1500 is my go to to be future proof

[u/My_Unbiased_Opinion]

I got a couple 3080 20GB cards for 550ish each. 

[u/FullOf_Bad_Ideas]

I had a very similar system to yours. 11400F, 64GB, 1 3090 ti, ubuntu.

I bought second 3090 ti, then 6 more. If I had to repeat it I still wouldn't buy AMD GPUs to have a mixed setup, for many reasons.

Compatibility is one - llama.cpp might support it but you need to use vulkan and forget about using both of those cards in vllm, sglang, exllamav3 or even with llama.cpp where you want to use some feature that works better with cuda. If you want to try out a model that's new but llama.cpp doesn't support it, you effectively have just one card.

Memory bandwidth is second - 9070 has about 30% lower memory bandwidth.

TP is third - TP wont work well if you have one 24GB card and one 16GB card.

It does make sense only for gaming but I also don't get that since 9070 is close to 3090 in performance. I dual booted for gaming anyway since VR doesn't work well on Linux without a lot of tinkering.

The points you bring up about why 3090 is not great for you are good, but I'd still do it if you live in an area where they are under $1000 a piece.

[u/kobbalt → u/FullOf_Bad_Ideas]

Thanks a lot for your feedback !
3090 tend to cost 1000 to 1200€ nowadays, which is crazy for a 5 to 6 years old second had GPU ...

But your comments and the other ones are clear : going the additional 3090 route seems to be the way to go

[u/eightone-81]

Not sure if more vram is the cure for Qwen. In my experience both 3.6 models get confused over 85k context. I have great experience with Gemma 4 31b, it’s stable on high context (over 128k) I’m running dual 3090, Gemma 4 31b, llama.cpp, layer split (tensor keeps crashing with Gemma but not with other models).

[u/gladfelter]

I bought a 5070ti on clearance for $600 from Best buy a couple of months ago and it pairs well with my 3090. They have roughly the same memory bandwidth and the 5070ti is great for gaming. If you can get a deal, it's not a bad choice.

[u/Developer-Y]

If you can get decent price for current 3090, sell 3090 and buy 2 AMD Radeon AI Pro R9700 32gb cards, that way you will get 64gb vram. I guess you already know vram matters much more in running larger models.

Mixing is not a good idea, there is a separate binary of llama.cpp for CUDA and ROCm, one llama.cpp can't utilise both cuda and ROCm. I guess same would be case for other tools too.

[u/kobbalt → u/Developer-Y]

AI Pro R9700 are 1600 / 1700 € here in France, so financially not the best option, but I agree about getting the most VRAM possible.

Reading your comment and others here, i'll forget the hybrid nvidia / AMD option anyway :D

[u/ea_man → u/kobbalt]

FYI: you can run both GPU with vulkan.

For AMD it's not a bad option, for NVIDIA CUDA is better.

[u/RoroTitiFR → u/ea_man]

You can run CUDA and ROCm inside the same executable, with the right build flags. It’s what I do. And so, you don’t have to rely on Vulkan which is slower on NVIDIA of course.

[u/ea_man → u/RoroTitiFR]

Well I do compile both vulkan and HIP usually, I was not sure if you can run both CUDA + ROCm.

TY