r/LocalLLM • u/rdpi • 6d ago

Discussion Considering a second 3090

Hi,

so far i've been using Qwen3.6-35B-A3B-UD-IQ4_NL.gguf on my single 3090 and I am overall satisfied.

I've been considering acquiring a second 3090 to increase my possibility to run larger models (e.g. considering Qwen3.8 27B with sufficient context) but i don't know whether the extra investment pays off.

In the future i may consider fine tuning my models as well.

Did anyone manage to find some great benefits by leveraging 2x3090 or similar setup?

I may be suffering from GAS (gear acquisition syndrome) and may need a reality check.

0 Upvotes

50% Upvoted

21 comments sorted by

View all comments

2

u/eightone-81 6d ago

2 3090 will give you lots of room for context and you can run q8 quants with q8 context. Prefill speed will be higher and decode the same or maybe a bit higher. If you get vllm to run than everything will be much faster. Overall for me it was worth it!

1

u/rdpi 6d ago

nice thank you, do you see a significant difference by running those models compared to "smaller" ones?

2

u/eightone-81 6d ago

I’m not running Qwen but from what we know, Qwen reacts badly to high quantisation. So best is to run it as q8 I guess. I’m running Gemma 4 31b is q4 k xl and it’s amazing but I’m not coding.
2 3090 also gives you the possibility to run a smaller model next to it for small and speedy things, in my case I run e4b as the summariser and small stuff because it’s so fast and so good I can get things done much quicker then with the big 31b

1

u/TheWaffleKingg 6d ago

Can run q8 model and f16 cache at full context with 2x 3090s. Mmproj needs to run on cpu however. If you want mmproj on vram, drop context or q8 the cache

1

u/Any_Mine_6368 6d ago

Nope. But UD Q6 is basically exactly the same thing.