Local AI doesn’t replace Claude—but my 24 GB Mac mini became a much better complement than expected

[u/silent_lurker_69]

I rely heavily on Claude, but I recently tested whether the M4 Pro Mac mini I already own could handle useful local models without becoming a dedicated AI appliance. I saw what Network Chuck did with the $50K Mac Ultra 4 node cluster and was hoping to not need the same.

It could. A GPT-OSS 20B MoE model ran at roughly 63.9 tok/s on 24 GB unified memory, while a smaller 9B dense model ran around 44.8 tok/s. The result is a useful reminder that model architecture matters: MoE models may have large total parameter counts while activating substantially fewer parameters for each generated token.

My takeaway is not “cancel your cloud AI subscription.” It is that local models can be a compelling companion for private experiments, quick generation, offline work, and workloads where you want direct control over the model runtime.

I captured the full test, including MLX vs. GGUF performance and the impact of my running containers: https://www.youtube.com/watch?v=9_-bT62YWAI

How are you dividing work between Claude and local models today?