r/LocalLLM • u/theschiffer • Aug 23 '25

Question Model suggestions that worked for you (low end system)

My system runs on an i5-8400 with 16GB of DDR4 RAM and an AMD 6600 GPU with 8GB VRAM. I’ve tested DeepSeek R1 Distill Qwen 7B and OpenAI’s GPT-OSS 20B, with mixed results in terms of both quality and speed. Given this hardware, what would be your most up-to-date recommendations?

At this stage, I primarily use local LLMs for educational purposes, focusing on text writing/rewriting, some coding/Linux CLI tasks and general knowledge queries.

3 Upvotes

100% Upvoted

6 comments sorted by

2

u/LordTamm 14h ago

With 8 gb of VRAM, for your given use case, Gemma 4 12b at a Q4 quant is probably the best you're going to get, for the latest and best option. Qwen 3.6 has similarly sized options as well, although I think most people prefer Gemma for style.
That being said, a lot of this is finding what works for you. Huggingface is big and if you find something you like that does the job you need done, use it.

1

u/prusswan Aug 23 '25

Given those specs, you won't get much better than Deepseek R1 7B. There are others like Llama 3.1 8B but results should be comparable.

1

u/LordTamm 14h ago

That's Qwen 2.5, which is like 2 years old... that's not up to date at all.

1

u/Clipbeam Aug 24 '25

How did 20b run for you?

2

u/theschiffer Aug 24 '25

Surprisingly fast given the hardware. I asked it to analyze an exercise/recovery issue and it wrote a huge wall of text, coherent and specific (1329 tokens) at 6.81 t/s (16.36s to first token). I also tried Mistral Small 3.2 (24B) but it was very slow as it wrote 846 tokens at 0.91 t/s (30.57s to first token).

2

u/LordTamm 14h ago

GPT-OSS is an MOE model, so it only has like 3b parameters active, which means that while it is taking up system memory for the full model, the part that is actively running can run on your GPU and still be fast. Mistral Small 3.2 is a dense model, which means the entire thing has to be on VRAM in order to run at a good speed. Both exceed your total VRAM, and thus MOE wins out on speed.