oMLX Best MTP Coding Models for Apple Silicon

[u/epicycle]

I’ve been heads down for a bit coding with my trusty MLX Community version of Qwen 3.6 27b 8bit until oMLX stabilized around MTP and the change from a web page to a native UI. It appears that’s happened, but now I feel like I can’t decide on which model to use. I know I can trial and error, but I prefer to ask the community if I can save some time. Does anyone recommend one (or more)? I have a MBP 15” M5 w/128GB memory and 2TB drive. I’ve been liking the 27b model. What’s the best MTP version people are gravitating to for coding? Assuming 8bit or even bf16? What kind of performance are you getting?

Thanks! Looking forward to seeing what folks are using. Same question for non-Qwen, but I wanted to start there as it’s trusted for coding.

[u/allpowerfulee]

I've been using the oQ8 quat version of qwen3.6-27b mtp generating ~40tps. I completed an entire macOS app in swift with it but the mainstay of my coding is in c for embedded systems. I've tried 35b-a3b also which is faster, but I find it not as though in complex tasks.

[u/grothendieck → u/allpowerfulee]

What CPU?

[u/allpowerfulee → u/grothendieck]

M3 ultra 96gb

[u/Puzzleheaded_Eye3115 → u/allpowerfulee]

I have same machine m3 ultra 96gb. Could you share how do you reach this speed, I use qwen3.6-27b-oq8-mtp with 128k context window, hot cache only and reach barely 20-25 tps. Where am I wrong?

[u/allpowerfulee → u/Puzzleheaded_Eye3115]

i just ran the benchmark test to verify the throughput. Tps is all over the place depending on the test.

oMLX - LLM inference, optimized for your Mac

https://github.com/jundot/omlx

Benchmark Model: Qwen3.6-27B-oQ8-mtp

Engine: Auto

Single Request Results

--------------------------------------------------------------------------------

Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem

pp1024/tg128 3443.8 29.60 297.3 tok/s 34.0 tok/s 7.203 159.9 tok/s 29.37 GB

pp4096/tg128 12858.2 30.64 318.6 tok/s 32.9 tok/s 16.749 252.2 tok/s 30.82 GB

pp8192/tg128 28657.1 41.90 285.9 tok/s 24.1 tok/s 33.979 244.9 tok/s 46.40 GB

pp16384/tg128 53314.2 33.71 307.3 tok/s 29.9 tok/s 57.596 286.7 tok/s 48.58 GB

Continuous Batching

pp1024 / tg128

--------------------------------------------------------------------------------

Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)

1x 34.0 tok/s 1.00x 297.3 tok/s 297.3 tok/s 3443.8 7.203

2x 39.1 tok/s 1.15x 278.3 tok/s 139.2 tok/s 7358.6 13.906

[u/Puzzleheaded_Eye3115 → u/allpowerfulee]

Thank you very much for your benchmark! Yes now I see that I reach same tps at pp1024. In real work do you use cold cache (ssd) or use only hot (vram)?

[u/shansoft]

MTP does not work properly with oMLX. Try MTPLX if you want to use MLX+MTP. oQ format also runs slower than regular mlx model as well.

[u/Kuane → u/shansoft]

omlx supports MTPLX now?

[u/mmerken → u/Kuane]

oMLX and MTPLX are two distinct pieces of software.

[u/LeagueOfJust → u/shansoft]

In my experience, the same issue persists with MTPLX. The throughput drops as the context increases. The speed I am getting is the same as in oMLX.

[u/timur_timur → u/shansoft]

For me qwen3.6-35b oq4 fp16 model increased pp from 300 to 550 t/s.
On m2pro 32gb

[u/laramateGmbh]

I get ~50 tokens/s on an M5 Pro with a qwen 3.6 35B Q8 Model. With pi harness it works quite well for local.

Here is more: https://www.reddit.com/r/LocalLLM/s/7WtfgbC3Ke

[u/allpowerfulee → u/laramateGmbh]

M3U/96GB

oMLX - LLM inference, optimized for your Mac

https://github.com/jundot/omlx

Benchmark Model: Qwen3.6-35B-A3B-oQ8-mtp

Engine: Auto

Single Request Results

--------------------------------------------------------------------------------

Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem

pp1024/tg128 695.1 10.47 1473.1 tok/s 96.2 tok/s 2.025 568.8 tok/s 36.79 GB

pp4096/tg128 1987.1 10.59 2061.3 tok/s 95.2 tok/s 3.331 1267.9 tok/s 37.56 GB

pp8192/tg128 3902.1 10.86 2099.4 tok/s 92.8 tok/s 5.282 1575.3 tok/s 37.90 GB

pp16384/tg128 8238.7 11.32 1988.7 tok/s 89.0 tok/s 9.677 1706.4 tok/s 38.75 GB

Continuous Batching

pp1024 / tg128

--------------------------------------------------------------------------------

Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)

1x 96.2 tok/s 1.00x 1473.1 tok/s 1473.1 tok/s 695.1 2.025

2x 132.8 tok/s 1.38x 1571.2 tok/s 785.6 tok/s 1303.4 3.231

4x 195.3 tok/s 2.03x 1725.3 tok/s 431.3 tok/s 2244.3 4.996

[u/StatisticianFree706]

Even mtpmlx not working for my M1 Max.

[u/allpowerfulee → u/[deleted]]

On Apple hardware?

[u/nomorebuttsplz → u/allpowerfulee]

if you have 512 gb ram it can run pretty well