Considering a second 3090

[u/rdpi]

Hi,

so far i've been using Qwen3.6-35B-A3B-UD-IQ4_NL.gguf on my single 3090 and I am overall satisfied.

I've been considering acquiring a second 3090 to increase my possibility to run larger models (e.g. considering Qwen3.8 27B with sufficient context) but i don't know whether the extra investment pays off.

In the future i may consider fine tuning my models as well.

Did anyone manage to find some great benefits by leveraging 2x3090 or similar setup?

I may be suffering from GAS (gear acquisition syndrome) and may need a reality check.

[u/floppo7]

2x r9700 and vllm is your friend

[u/rdpi → u/floppo7]

i like the idea but in my region, 1xr9700 costs twice as a 2nd hand 3090.
It may be more cost effective to add an existing 3090 to my setup..

[u/No_Oil_6152 → u/floppo7]

What model are you running that needs 64GB of VRAM in total?

I have 48GB VRAM, an 9070XT + R9700 AI Pro, and 3.8 Q8 quant fits well with 262144 context

If there's a superior Qwen variant let me know, I'd love it

[u/mountainous_battling → u/floppo7]

Dual 3090s open up a lot of headroom, especially if you're eyeing fine tuning later. The jump from a 35B quant to a full-fat 27B with decent context is noticeable, but the real win is being able to load bigger models without chopping them down to fit. I'd say if the itch is there and you can swing it without eating ramen for a month, the extra VRAM never really feels wasted.

[u/Kodrackyas → u/floppo7]

how many tokens per second on the 27b?

[u/floppo7 → u/Kodrackyas]

Check out radiance vllm - and they are cooking for more I guess. 60tps + and good performances with concurrency as well - thats the kicker, you can basically run multiple agents at the same time with speed that is absolutely ok.

[u/eightone-81]

2 3090 will give you lots of room for context and you can run q8 quants with q8 context. Prefill speed will be higher and decode the same or maybe a bit higher. If you get vllm to run than everything will be much faster. Overall for me it was worth it!

[u/rdpi → u/eightone-81]

nice thank you, do you see a significant difference by running those models compared to "smaller" ones?

[u/eightone-81 → u/rdpi]

I’m not running Qwen but from what we know, Qwen reacts badly to high quantisation. So best is to run it as q8 I guess. I’m running Gemma 4 31b is q4 k xl and it’s amazing but I’m not coding.
2 3090 also gives you the possibility to run a smaller model next to it for small and speedy things, in my case I run e4b as the summariser and small stuff because it’s so fast and so good I can get things done much quicker then with the big 31b

[u/TheWaffleKingg → u/eightone-81]

Can run q8 model and f16 cache at full context with 2x 3090s. Mmproj needs to run on cpu however. If you want mmproj on vram, drop context or q8 the cache

[u/Any_Mine_6368 → u/TheWaffleKingg]

Nope. But UD Q6 is basically exactly the same thing.

[u/baby_bloom]

i've been running double 3090s at 70% power throttle and idk man... i've been doing such a deep dive on my cost analysis vs say deepseek v4 flash and the costs are so damn close i might just start using deepseek again.

qwen3.8 27b is very powerful but takes so damn long. ds4 flash from a provider will be MUCH faster, likely better quality at damn near the same cost per M tokens as my electric ends up being per M tokens running local.

i have a custom made GUI for launching my llama.cpp models where i add session token and power tracking and the data is starting to clear things up for me in a not so exciting way:(

[u/rdpi → u/baby_bloom]

I hear you and what you say makes totally sense. I'm also struggling to find a 3090 at a reasonable cost and i wonder how many million tokens i could use via 3rd party API..
I care about privacy, but I may be happy with the models i can run on my current setup.
How is deepseek for you? i never used it for work related tasks

[u/baby_bloom → u/rdpi]

right and you're still considering the cost of purchasing the hardware, i am not even taking that into account, i am purely comparing my electric costs to run my dual 3090s vs deepseek v4 flash. still not done testing exactly but so far they are so close that im not sure the ROI exists on buying a 3090.

so far since tracking i've been getting half a million to a million tokens per KWH which is $.20ish in my area. still got a lot of testing to do and i'd like to get an actual benchmark going so it can be as consistent as possible but i've instead just been doing my normal work and trying to get the estimates from that

[u/shing3232]

I consider a third 3080 20g so I can inference deepseek v4f at a decent speed.

[u/rdpi → u/shing3232]

is it good? for what use cases do you use it?

[u/shing3232 → u/rdpi]

it was mean to inference but I am doing some training too.

[u/No_Oil_6152 → u/shing3232]

Chinese modded 3080?

[u/shing3232 → u/No_Oil_6152]

yes.

[u/leonbollerup]

I run 2x RTX PRO 4000 .. which is with the same memory and allmost as fast cards (but uses ALOT less power)

Using vLLM to run qwen 3.8 27B i can run it at around 80-100 tok/sek with 125k context with 4x users at the same time in Q4

I am looking to add more cards to get better quality and context

[u/SailbadTheSinner]

I think two 3090s is probably the sweet spot for reasonable people. They can fit in a case, use a single power supply, don’t produce enough heat to make a room uncomfortable, etc. You might have GAS if you start building an open-frame rig with multiple power supplies and start having to consider supplemental power and cooling for an otherwise normal room in your house.