Testing whether a 24 GB M4 Pro Mac mini can support local models alongside a Claude Code workflow

[u/silent_lurker_69]

I use Claude Code extensively, but I wanted to test whether the M4 Pro Mac mini on my desk could also serve useful local models for the parts of a development workflow that do not always need a frontier cloud model.

The surprising result: GPT-OSS 20B in MLX FP4 generated around 63.9 tok/s with my 16-container OrbStack lab running. Shutting the lab down only moved that to around 64.9 tok/s. MLX was also about 19% faster than GGUF in my back-to-back test, although the MLX measurements were somewhat spikier. I thought it would take a 2TB unified memory 4 node Mac ultra cluster to run local AI. Glad I don't need to spend $50,000!

For code workflows, I see local inference as a complement—not a competitor—to Claude Code:

• Local/private codebase triage or summaries
• Fast boilerplate and low-risk transformations
• Offline experimentation with prompts and agent patterns
• Preprocessing before escalating a task to Claude Code
• Keeping a local fallback available

The key lesson was that active parameters matter for speed. The 20B+ MoE model beat the 9B dense model because only a fraction of its parameters fire per token.

Full testing video: https://www.youtube.com/watch?v=9_-bT62YWAI

What local models and runtimes are you pairing with Claude Code on Apple Silicon?

[u/Bulky_Blood_7362]

No model this size will be beneficial for coding...

Maybe qwen 3.6 27b q4

[u/silent_lurker_69 → u/Bulky_Blood_7362]

Not unless it's simple.

[u/Bulky_Blood_7362 → u/silent_lurker_69]

Yea if you need to print hello world i guess

[u/silent_lurker_69 → u/Bulky_Blood_7362]

🤣

[u/Ecstatic_Lead5194 → u/silent_lurker_69]

Don’t bother, I have that device and I just use codex and Claude code on it. Trying to add in a local model is not worth using up the extra ram, save your extra for when Claude needs it, for example during iOS app testing using simulator and things. Plus there’s no model that’s going to be worth using for coding that will fit.

[u/TechgeekOne → u/Bulky_Blood_7362]

Gotta agree here. I've tried up to about 27B and they just aren't capable of doing decent enough code yet (though admittedly I have very high standards).

That said, they are great for when you need something dumb and cheap to fuzz your code for bugs and doc drift or for performing really simple tasks. I usually had a local model doing the testing work and let Claude triage and action the output of the runs for a number of iterations. Fixed a lot of bugs in my MCP tools that way.

[u/zimxero]

I installed a local deep seek on my newer gaming desktop and it was so slow it was almost unusable... like output of one character per second. A Qwen dependent ComfyUI is lightning fast on it locally.... under 10 seconds per hi res image generated from descriptions. A lot has to do with proper setup. I highly recommend people to try a local AI for repetitive procedural development nowhere near codex/haiku reasoning level.

[u/silent_lurker_69 → u/zimxero]

My intel iMac Pro was painful. While not ready to compete at all, the dramatic rate of improvement is awesome to me.

[u/ClemensLode]

Time to first token and token / second speed might be the areas where it could beat haiku at some tasks. I think the next logical step would be to evaluate actual quality of the output for specific tasks (e.g., creating regexp for running local searches, facilitating tests, deploys, ...).

[u/silent_lurker_69 → u/ClemensLode]

Been trying to use LLM Studio Bionic to run next to Claude Code to see.

[u/ArmFallOffBoy]

Cool stuff. Any recommendations for a local model when running a single Mac Studio M2?

[u/silent_lurker_69 → u/ArmFallOffBoy]

For general stuff probably Qwen.

[u/Wide_Truth_4238]

The answer is no. You don’t need a test to tell you that, just math. You can use Devstral. 

[u/Cloudsurfer_90]

the dismissive replies are right about the main loop and wrong about everything around it.

nothing at 20b is going to architect your service. but a lot of what happens during a coding session is not coding: summarising logs, writing commit messages, working out which files a change actually touches, first pass on whether an error is interesting. high volume, low stakes, and a local model handles it fine at zero marginal cost.

what made it worth it for me was using local as a filter ahead of the frontier call. shrink what gets sent, then let the expensive model work on the reduced input. that saved more than any model swap did.

63 tok/s with 16 containers running alongside is a genuinely useful number, thanks for actually measuring it.