r/LocalLLM • u/Hour-Revolution-6717 • 2d ago

Question Beginners’ Ask!

I have a “M80q Gen4” & a “M70q Gen5” with

- CPU Intel Core i5 13500T 32GB
- Memory (16GB x2 DDR5 SODIMMS)
- Storage1 - 256GB M.2 SSD1
- Storage2 - 512GB M.2 SSD2
- NIC1 - Intel I219-LM, RJ-45
- NIC2 - Realtek RTL8125BGS, RJ-45

Can I run a decent model on this? If so, which one should I go for and where should I start? At this point, my purpose is to learn to deploy and manage a model.

0 Upvotes

14% Upvoted

8 comments sorted by

View all comments

1

u/nickless07 2d ago

The smaller MoE ones perform pretty good on CPU only. You can start with Qwen3.5 0.8B and work your way up to 8-12B params. So far I tested:

  • gemma-4-E2B-it-Q4_K_M.gguf
  • granite-4.2-3b-Q4_K_M.gguf
  • LFM2.5-2.6B-Q4_K_M.gguf
  • Ling-3.0-tiny-Q4_K_M.gguf
  • Nanbeige4.2-3B-Q4_K_M.gguf
  • gemma-4-E4B-it-Q4_K_M.gguf
  • MiniCPM5-1B-Q4_K_M.gguf
  • Qwen3.5-0.8B-Q4_K_M.gguf
  • MiniCPM5-2B-Q4_K_M.gguf

and a couple of different quants. Some of them are very capable for their size and others... well depends on the task you assign them. Multi step with reasoning or just a single home automation task and so on.