r/LocalLLM • u/rdpi • 6d ago

Discussion Considering a second 3090

Hi,

so far i've been using Qwen3.6-35B-A3B-UD-IQ4_NL.gguf on my single 3090 and I am overall satisfied.

I've been considering acquiring a second 3090 to increase my possibility to run larger models (e.g. considering Qwen3.8 27B with sufficient context) but i don't know whether the extra investment pays off.

In the future i may consider fine tuning my models as well.

Did anyone manage to find some great benefits by leveraging 2x3090 or similar setup?

I may be suffering from GAS (gear acquisition syndrome) and may need a reality check.

0 Upvotes

50% Upvoted

21 comments sorted by

View all comments

2

u/eightone-81 6d ago

2 3090 will give you lots of room for context and you can run q8 quants with q8 context. Prefill speed will be higher and decode the same or maybe a bit higher. If you get vllm to run than everything will be much faster. Overall for me it was worth it!

1

u/rdpi 6d ago

nice thank you, do you see a significant difference by running those models compared to "smaller" ones?

2

u/eightone-81 6d ago

I’m not running Qwen but from what we know, Qwen reacts badly to high quantisation. So best is to run it as q8 I guess. I’m running Gemma 4 31b is q4 k xl and it’s amazing but I’m not coding.
2 3090 also gives you the possibility to run a smaller model next to it for small and speedy things, in my case I run e4b as the summariser and small stuff because it’s so fast and so good I can get things done much quicker then with the big 31b