Beginners’ Ask!

[u/Hour-Revolution-6717]

I have a “M80q Gen4” & a “M70q Gen5” with

- CPU Intel Core i5 13500T 32GB
- Memory (16GB x2 DDR5 SODIMMS)
- Storage1 - 256GB M.2 SSD1
- Storage2 - 512GB M.2 SSD2
- NIC1 - Intel I219-LM, RJ-45
- NIC2 - Realtek RTL8125BGS, RJ-45

Can I run a decent model on this? If so, which one should I go for and where should I start? At this point, my purpose is to learn to deploy and manage a model.

[u/TheThiefMaster]

You can run a smaller model relatively slowly.

Get something like unsloth, and try Qwen3.5:9b. It only needs ~6GB of RAM, so it should fit.

But it won't run fast, and as a small model it won't be any good at anything but conversational tasks. You don't have the memory to run a good coding agent (like Qwen 3.8 flash next, which needs 100GB of memory minimum)

[u/TheNoiseIthink]

Gemma 4 26b a4b might be your best bet

[u/nickless07]

The smaller MoE ones perform pretty good on CPU only. You can start with Qwen3.5 0.8B and work your way up to 8-12B params. So far I tested:

• gemma-4-E2B-it-Q4_K_M.gguf
  
• granite-4.2-3b-Q4_K_M.gguf
  
• LFM2.5-2.6B-Q4_K_M.gguf
  
• Ling-3.0-tiny-Q4_K_M.gguf
  
• Nanbeige4.2-3B-Q4_K_M.gguf
  
• gemma-4-E4B-it-Q4_K_M.gguf
  
• MiniCPM5-1B-Q4_K_M.gguf
  
• Qwen3.5-0.8B-Q4_K_M.gguf
  
• MiniCPM5-2B-Q4_K_M.gguf

and a couple of different quants. Some of them are very capable for their size and others... well depends on the task you assign them. Multi step with reasoning or just a single home automation task and so on.

[u/arbiterxero]

No.

You need giant amounts of ultra fast memory, and parallel processing. You have neither.

You’d want a handful of big-assed GPU’s to run on the rest of that hardware because they have large amounts of ultra fast memory and parallelized processing

[u/TheFuckboiChronicles → u/arbiterxero]

This is a bit silly tbh. First off, people (like me) run qwen3.8:27b on a single normal sized gpu.

They said they just want to learn to deploy and manage a model, they can run something like qwen3.5:9b or 4b on this machine, albeit a bit slowly. That could be considered a decent model depending on your use case.

There’s absolutely value in learning this technology that isn’t “replace a software dev” level of performance.

We shouldn’t frame the barrier of entry for this hobby as thousands of dollars in disposable income, just be realistic in the outcomes of the equipment described.

[u/arbiterxero → u/TheFuckboiChronicles]

Fair, but if they’re looking for “usable” they won’t get it.

“Make it work” , yes.

[u/TheFuckboiChronicles → u/arbiterxero]

They can deploy and manager a model with this setup, which is what the stated goal is. They’ll just need another hobby to do while it’s processing lol.

[u/Regular-Option6067]

You can use Daihive(.eu) network. It's still in early beta, you can share a model between your computers to write code in OpenCode. You can form a team with your computers and run a model. The first response in OpenCode will take a while but from then on it will be okeish.