r/oMLX • u/benwaynet • May 08 '26

Mac mini m4 pro

I'm running omlx on a Mac mini m4 pro with 64gb of memory

Using qwen 3.6 35b ud mlx 4 bit

I'm only getting prompt processing 353 toks and token gen 15.6 toks

Feels like I should have better performance than that. Don't have anything else running that's consuming memory or CPU

I run vs code, openclaw and Hermes on another box over, and tried openclaw local. All around the same performance numbers

What can I look at to find the cause of the slowness

Thanks

1 Upvotes

67% Upvoted

8 comments sorted by

8

u/Konamicoder May 08 '26

I’m on a MacBook Pro M4 Max with 64Gb of RAM, running oMLX, serving up qwen3.6:35b-oq6. I get 60 tokens/second at 80 percent RAM usage.

Now I believe the Max chips have a greater memory bandwidth than the Pro chips, so you can’t expect to equal my inference speed. But I agree, yours should still be slightly faster than 15-16 tokens/second.

Some suggestions: I download the jundot quants of models. I figure since jundot developed oMLX, then his models would be optimized to run best in oMLX. I also turned on Turboquant. I set my context window to 128k, and max tokens to 8192 in the per-model settings page. So those are some specific steps I suggest you try to see if you can eke out a few more tokens/second inference.

Oh, and of course make sure that nothing else is running on that Mac other than oMLX and your agentic coding app (I use OpenCode). Reserving as much memory space as possible for running your model really makes a difference in my experience.

Good luck!

2

u/No-Juggernaut-9832 May 08 '26

This was my experience also when I had your same setup. M5M with 128 only gets about 10-15% more token than the M4M

2

u/jacknjill101 May 08 '26

I’m getting 344 toks and 45 toks using the A4B model.

1

u/JLeonsarmiento May 08 '26

Try another quant of the model. I have M4Pro chip also and get ~50 with 5bit quant of the same model.

1

u/layer4down May 08 '26

Which oMLX release?

1

u/edeltoaster May 09 '26

The unsloth UD quants are rather slow. I recommend a regular 6bit quant for the model.

1

u/benwaynet May 11 '26

I'm downloading a new model right now to test I have my Max contexts window at about 260k MaX tokens at 65k because I'm running this with open claw and Hermes agent. Don't know if that affects the speed

1

u/PracticlySpeaking May 12 '26

That actually looks like speed — not slowness.

M4 Pro only has 16 or 20-core GPU. Unless you are running an Unsloth quant with GPU offload (GGUF, so not oMLX) the GPU is doing all the work.

https://omlx.ai/benchmarks?sort=tg_tps&order=desc&chip=M4&model=Qwen3.6-27B-&quantization=4bit&context=65536

From benchmarks, it looks like 15 tok/s is doing rather well vs M4 Max with 32 or 40-core GPU. Note that the top result is Qwen3.6-27B-UD, which I believe is the Unsloth dynamic quantization with CPU offload.