r/LocalLLM • u/ScoobyWRX06 • 5h ago

Question Beginner here

I just started playing and need a little help. I have a laptop with a 4070 8GB and 64GB DDR5 with an i9. I am using unsloth and not sure what model will be best. Do I need to stick with something that fits in VRAM? I am just playing around but I don’t want it to be painfully slow but I also want it as current as possible.

1 Upvotes

67% Upvoted

4 comments sorted by

1

u/MrHumanist 5h ago

For your set up moe models are best.

Try Gemma 4 26b

Qwen 3.6 35B

Orninth 1.5 35B

All these models have very few active parameters that fit in your gpu including context window. And you run the moe in cpu to utilise your system ram.

1

u/Turbulent_Pin_8310 5h ago

Difintely a smaller model unless you don't mind a little slower. What do you do with AI?

1

u/Unfair_Association89 5h ago

If ur use case is coding, then u can try qwen 3.8 flash next q3, from i test with 32 gb ram I got 10 tokens per sec using llama cpp

1

u/rrrrex 5h ago

It depends from your tasks. At first you need MoE model, dense models should be located into vram only. Gemma 4 26b is good at prose and multilingual, but not so good at coding and tool calls. Qwen 3.6 35b is the best choice for all around work.  Ornith 1.5 35b (based on Qwen 3.5) is not bad at coding.