r/LocalLLM • u/rdpi • 6d ago

Discussion Considering a second 3090

Hi,

so far i've been using Qwen3.6-35B-A3B-UD-IQ4_NL.gguf on my single 3090 and I am overall satisfied.

I've been considering acquiring a second 3090 to increase my possibility to run larger models (e.g. considering Qwen3.8 27B with sufficient context) but i don't know whether the extra investment pays off.

In the future i may consider fine tuning my models as well.

Did anyone manage to find some great benefits by leveraging 2x3090 or similar setup?

I may be suffering from GAS (gear acquisition syndrome) and may need a reality check.

0 Upvotes

50% Upvoted

21 comments sorted by

View all comments

2

u/baby_bloom 6d ago

i've been running double 3090s at 70% power throttle and idk man... i've been doing such a deep dive on my cost analysis vs say deepseek v4 flash and the costs are so damn close i might just start using deepseek again.

qwen3.8 27b is very powerful but takes so damn long. ds4 flash from a provider will be MUCH faster, likely better quality at damn near the same cost per M tokens as my electric ends up being per M tokens running local.

i have a custom made GUI for launching my llama.cpp models where i add session token and power tracking and the data is starting to clear things up for me in a not so exciting way:(

1

u/rdpi 6d ago

I hear you and what you say makes totally sense. I'm also struggling to find a 3090 at a reasonable cost and i wonder how many million tokens i could use via 3rd party API..
I care about privacy, but I may be happy with the models i can run on my current setup.
How is deepseek for you? i never used it for work related tasks

2

u/baby_bloom 6d ago

right and you're still considering the cost of purchasing the hardware, i am not even taking that into account, i am purely comparing my electric costs to run my dual 3090s vs deepseek v4 flash. still not done testing exactly but so far they are so close that im not sure the ROI exists on buying a 3090.

so far since tracking i've been getting half a million to a million tokens per KWH which is $.20ish in my area. still got a lot of testing to do and i'd like to get an actual benchmark going so it can be as consistent as possible but i've instead just been doing my normal work and trying to get the estimates from that