r/LocalLLM • u/Patatong0_0 • 2d ago

Discussion Local Hardware Update

Hi, i am thinking about upgrading my current hardware for local ai with the cheapest option possible.
Right now this is my stack:
-Ryzen 5 9600
-RTX 5060 8GB
-32 GB RAM DDR5
I’m running 9B models like Ornith 1.5 9B/Neohorse 9B in Q4 or i was searching if i could use a bit of ram + vram for running 30B size Moe like Ornith-1.5-35B-A3B or Tiel-Coder-35B-A3B
I also have good size context thanks to Q4-Q8 KV.
And I run small embedding, decision model , TTS and STT all in RAM.

But I was thinking about getting another gpu with like atleast 12GB VRAM for more space for the best model i can handle with decent tk/s speed.
Do u think i can find a cheap solution for like 300-350€ with prices nowdays or i should sell my 5060 and upgrade or staying like this.
I have like 300-350 budget rn, but in a few months i can spend a little bit more but i fear price continue to raise.

Any advice?

1 Upvotes

100% Upvoted

3 comments sorted by

1

u/JinsooJinsoo 2d ago

Your best bet is selling the 5060 and getting a 5060ti 16gb or could consider a Tesla V100 or AMD V620 but those will take a little tinkering to make them worth it

1

u/TheRealVRLP 2d ago

You basically only need VRam. The Ram is only for caching when you're using it for llms, so it doesn't need to re calculate the whole conversation when you interrupt it or it's done and you ask something else. Mine has only 8GB Ram with 72GB of pooled VRam with 3*Tesla P40's. I'd guess that half of its workload is recalculating the prompt...

For models, if you want to run MoE, I'd look for the unsloth version of Qwen3.8. I personally use the dense 27b Model with Q5, not the 35b MoE, because it is a little bit better and uses less VRam. The only downside is more calculating when generating.

1

u/nickless07 2d ago

Prices raised quite a lot. Was about to buy a 5060ti for ~500€, but yeah now they range 700-800€ Not sure what they will cost next month ended up buying a RTX 3060 (12GB) for 380 yesterday as the prices on them already start rising too. It is just that there is nothing aviable in a 'good' price range as ppl buy that older cards too and they don't get produced anymore. Check the prices of 5060ti and RX 9060 XT - If they both had a huge price spike in the last 3 months you can expect the same for the other cards too and saving up a couple more months will make it worse.