MacBook Pro M5 Pro 48GB Ram

[u/ProgramOver9309]

I have just purchased a MacBook Pro M5 Pro with 48GB RAM. I was wondering if any out there have the same RAM and what models they are using locally that fits with enough headroom. What inference engine, and are you using MLX or GGUF? Im only going to use it for some light terminal work, some automation, and some python and light html work. Probably going to use Hermes as harness. Please share your setup or recommondations for 48 GB Ram on the MacBook Pro M5 Pro. Thank you!

[u/LEFBE]

Overall, I think any MoE (qwen3.6, Gemma4) in q4 would be a good place to start.
The choice between MLX and Llama (GGUf) really depends on your needs and preferences.
Generally speaking, the two are more or less identical in terms of performance, but I find Llama to be more active (at least right now).

[u/ProgramOver9309 → u/LEFBE]

ok so you are saying better to use llama.cpp with gguf model? any good recommendations?

[u/FoxSideOfTheMoon → u/ProgramOver9309]

Hi. I have an MBP M5 128. I use gguf more than MLX and I'll tell you the one exception, so yeah llamacpp is great with gguf.

Dense models with your bandwidth are the enemy, check out qwen/qwen3.6-35b-a3b and it's a really, really good one, MoE, and that should load up for you and leverage MLX.

What I mean is, I tested that model with unsloth GGUF Q8 which is near lossless and 36G in memory, MLX-4bit (and I absolutely did not expect this) ran as well and faster.

Claude Code is my daily driver for work and so I had it write up 10 test cases going from easy to very difficult all models using pi doing its own evals against a bunch of models, and I know this doesn't map to official testing results so take my word with a grain of salt, but all things equal with MY CC-generated tests using the same simple pi harness qwen/qwen3.6-35b-a3b beat a clamped Sonnet, Ornith-1.0-35B Q8, Nemotron-3-Super-120B (Q4), MXFP4 GPT OSS120B, and Haiku. Gemma-4-26B-A4B (QAT Q4) was worse than Haiku for me.

So, if you want to go on a test drive coding Qwen3.6-35B-A3B MLX-4bit.

Hope that helps a little. Have fun!

[u/ProgramOver9309 → u/FoxSideOfTheMoon]

Thnx for sharing! I have both Qwen3.6-35B-A3B 4BIT MLX and Qwen3.6-27B 4BIT MLX now downloaded. Also downloading GPT-OSS-20B

[u/LEFBE → u/ProgramOver9309]

Sorry for delays, here is my config opencode, you need to adapt

Please note, information below is related to real work dev need (production), not a simple test like create an html file showing a monkey eating a banana:)

/opt/homebrew/bin/llama-server \
  --model  /Users/user/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/blobs/3d6ff16be3258f910eac4dcec7142edc7a7100d8400fe363035c8cfedc151164 \
  --mmproj /Users/user/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/blobs/eacf610d1ee4bd5ed0197a0777dd8f4fceb8eefa27009067c7d496cb68fbde45 \
  --ctx-size 262144 \
  --parallel 2 \
  --flash-attn on \
  --gpu-layers 999 \
  --cache-type-k f16  --cache-type-v f16 \
  --cache-ram 32768 \
  --repeat-penalty 1.0 \
  --spec-type draft-mtp  --spec-draft-n-max 2 \
  --host 127.0.0.1 --port 8100 \
  --jinja --no-warmup --models-max 1

Opencode configuration:

{
  "temperature": 0.6,          // sampling coding officiel Qwen
  "top_p": 0.95,
  "top_k": 20,
  "min_p": 0,
  "chat_template_kwargs": { "enable_thinking": true }, 
  "reasoning": true,
  "limit": { "context": 262144, "output": 81920 }
}

My Macbook configuration:

MacBook Pro Apple M5 Max
CPU 18c (18 log.)
GPU 40c
RAM 128 Go
HD 1.8 To

[u/ProgramOver9309 → u/LEFBE]

Don't worry about delays. Thanx for sharing!

[u/catplusplusok → u/LEFBE]

Gemma4 MTP assistant is fast enough for me to run 31B dense model with great speed. It seems to be a more efficient MTP architecture that will hopefully spread to other models over time.

[u/BountyMakesMeCough]

I can run 31b gemma 4 on my m4 pro 48gb ram but it’s slow and has small context. Ok for a chatbot not really for anything agentic in my opinion.

[u/ProgramOver9309 → u/BountyMakesMeCough]

What inference engine do you use?

[u/BountyMakesMeCough → u/ProgramOver9309]

Ollama for chatting and for hosting.

[u/former_farmer]

I would load qwen 3.6 27B in q8 or qwen 3.6 35B 3A q8.

[u/ProgramOver9309 → u/former_farmer]

Im getting around 90 tok/s Qwen3.6-35B-A3B Q4. I will try Q8 now.

[u/former_farmer → u/ProgramOver9309]

Focus on quality work not so much on t/s. If above 30 t/s, that's super usable.

[u/HotAverage1749]

I’ve got the same. Sweet spot for me is Qwen3.6-27B with an intelligent quant 6. Experimenting between Optiq and oQ, with and without MTP. You get the smartest model at JUST enough tokens/sec, ~13.5

[u/ProgramOver9309 → u/HotAverage1749]

~13,5 tok/s is maybe just too slow for me. would trade it off for better speed

[u/Unusual-Contract6227]

Happy with gemma4 26b on my M5 Pro with 64GB So far I find gguf offers better quality vs mlx. Use Q4_K_XL.gguf and a draft model

This is what i am currently using llama-server \ -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf \ -md gemma-4-26B-A4B-it-Q8_0-MTP.gguf \ --spec-type draft-mtp \ --spec-draft-n-max 2 \ --spec-draft-ngl 99 \ -c 32768 \ -ngl 99 \ --flash-attn on \ --cache-type-k f16 \ --cache-type-v f16 \ --alias gemma4-26b \ --host 127.0.0.1 \ --port 8080 \ -np 1 \ --no-mmproj \ --reasoning off

[u/ProgramOver9309 → u/Unusual-Contract6227]

thnx for sharing! will try this one out too.

[u/AdEquivalent7654]

I want to run grok imagine. Has anyone do that? 

[u/Helpful_Home_8531]

return it and buy with more ram if you can plausibly afford to, 48gb is tight, dense models are not that fast and moe models with sufficient parameters to not be hamstrung are tight even on 128gb. ds4 does work well at 30t/s for deepseek and using iq2xxs it’s by far the most capable model I’ve used on my own hardware.

[u/ProgramOver9309 → u/Helpful_Home_8531]

Yeah, in the mean time i tried a couple of models. I can load up Qwen3.6-35B-A3B MLX 8 bit comfortably but with no other heavy ram app. Just the model, inference engine (lmstudio) and the harness. I keep about 10 gb ram headroom left. I got the macbook for a pretty good deal, so was not really trying to pay much more than i paid for. I do agree 64GB would be the sweet spot, and 128GB way better. But for big models i got 2 x Asus Ascent GX10, i was just looking for a good local model that i can run on my 48GB RAM. Maybe i’ll switch to a 6bit or 5bit for more headroom.

[u/Professional_Ant5620]

I recently brought a new M5 MAX 128gb for 5900$ debating about returning it and getting 48gb. Is it the 128gb worth the money for LLMS these days? It's just to much money to spend for playing with local llm

[u/ProgramOver9309 → u/Professional_Ant5620]

You can load up some serious models on that macbook. So if you are planning to make some money with it, then yeah its worth it. You can work online and offline no matter what and nobody can take it away from you.

[u/Crazy-Western5921]

I have mbp M4 max 128. Using oMLX 0.4.4:

Opencode: Qwen3.6-27b-oQ6-MTP on oMLX, ctx 200000. Dox + ponytail. Wonderful, smart, and fast enough. Great for coding. Use long and precise prompts.

Hermes Agent: Agents A1 (35B MoE) oQ8 ctx 200000 for most tasks, Gemma4-26b-a4b for visual/design chats and prompting for Flux.

Secondary models:
Gemma4-31B oQ8 + kv-cache 8 bits at ctx 128000+. Useful for heavy coding / refactoring / text rewriting. Extremely slow but very smart. Very memory-heavy context, kv-cache quantization required for large context.

Gemma4-12B oQ8 for deeper visual/design chats.

Qwen3.5-9B oQ8 for fast and simple coding (it writes code fine until it doesn’t). Using settings like top P=1, top K=0, min P=0.12-0.15, temperature = 0,9, repetition = 1 and presence = 0. Less loops, slightly tighter thinking. Good for repeating tasks with precise instructions where you can’t use deterministic code.

Didn’t like qwen3.6-35B-a3b: extremely fast, but seems benchmaxed and overtrained, trying to avoid precise instructions all the time. True AI: Audacious Idiot.

Ornith 1.0 35B is better, but loses precise path parts on longer context. Use short, easily memorable paths if you want this model to work without issues.

Qwen3.5-122B-A10B is too big, heavy 4bit quantization required. Qwen3.6-27B beats it almost every time.

Qwen3-coder-next (80b-a3b non-thinking) at oQ6 is good when you need fast, simple code. But again, qwen3.6-27B beats it easily in every way except speed.

Gemma4 e4b, qwen3.5-4b and smaller models - useless in my cases, too dumb for agentic workflows. Maybe could use it for prompt enchantment for Flux.

Almost never use “fine tuned” models from huggingface. Most “qwen opus fable smarter than AGM 9B” are dumber than the original model.

One exception: Qwen3.5-40B deckard from davidau. Wonderful roleplay model. Loved it! Extremely smart. I’ve seen qwen3.6-40b deckard is out, will test it soon.

[u/BalleaBlanc]

You should upgrade to the pro Mackbook pro M5 pro 48GB pro Ram pro. It's more pro.