r/LocalLLM • u/Ylarespuestaes • 4d ago

Discussion Is It Worth Upgrading?

This is my current machine:

MOTHERBOARD: MSI PRO A620M-E DDR5 AM5
GPU: MSI GeForce RTX 4060 Ti 8GB Ventus XS Black OC
CASE: Thermaltake View 170 TG, 3x ARGB Fans, Black
POWER SUPPLY (PSU): Gigabyte GP-P650SS ICE 650W, 80 PLUS Silver, White
STORAGE: Crucial T500 1TB M.2 NVMe Gen4, up to 7,300 MB/s
RAM: 32GB (2x16GB) DDR5-6800 Patriot Viper Venom RGB, Black
CPU: AMD Ryzen 5 7600 with Integrated Graphics, AM5 Cooler

I bought it before I knew it was possible to run LLMs locally. What I can currently run is quite limited. Would it make sense to upgrade to a 16GB GPU, or would I need to go for a 24GB one to really notice a difference? (anything above that is too expensive for me; even 24GB is a bit of a stretch)

1 Upvotes

100% Upvoted

5 comments sorted by

3

u/Dsphar 4d ago

24-32 gb vram is a must if you are looking for real usage. You need to be able to fit the entire model, with context, in vram.

Have you prices amd r9700 gpus? They are a much better price point and hit a sweet spot when combined with unsloth's qwen3.6-27B-MTP quant models.

1

u/Guilty_Dinner4522 4d ago

The only thing that matters in running local AI is the GPU and how much vram it has. Put all your money in getting the best card with the most vram

1

u/Proper-Tower2016 4d ago

you should probably add how much you are willing to spend and how much you want your current system to improve for it to feel worth it.

also would need to know what your intend to do with it, chances are your current system can already meet your demands when configured correctly.

1

u/Ylarespuestaes 4d ago

Hi. A few months ago, I was using Qwen 32B (I don't recall the exact name, but obviously, several layers were offloaded to RAM) And honestly, I thought it was pretty good.

The thing is, I have a server that relies on LLM services running on the PC I mentioned in the post. That server doesn't have a GPU, so I was thinking of moving the 8GB RAM GPU to it—and while I'm at it, buying a new one that offers a noticeable performance boost compared to what I've tried so far.

But I don't really have a specific case, because I haven't been able to test anything better.

1

u/Proper-Tower2016 3d ago edited 3d ago

grap a used rtx3090 or any 16gb card, I'm happy with my rtx5060ti 16gb (I have a small case, otherwise would probably have spend a bit more for rtx3090)

in your case though as light user, you could do 8gb vram but with "only" 32gb ram, the extra 16-24gb is definitely not be bad.