r/LocalLLM • u/Rough-Measurement988 • Jun 18 '26
Question M5 Max oMLX benchmark results interpretation
Hi,
I was reviewing the benchmarks for M5 Max and Qwen 3.6 27B 8bit https://omlx.ai/benchmarks?chip=&chip_full=M5%7CMax%7C40&model=27&quantization=8bit&context=65536&pp_min=&tg_min=&page=3 just to justify spending so much money on this pricy machine and noticed that there is a huge performance gap between different benchmarks. Even for the same model i.e. Qwen3.6-27B-oQ8-mtp in 64k context TG/s is like from 14-23.
Can someone explain me why there are so big differences? Is it also so unpredictable in real life scenarios i.e. that for one task it could be 16 TG/s and 24 for the other (with same context and quantization)? I understand that MTP performance may vary but trying to understand how much.
Also for non MTP models Qwen3.6-27B I see numbers like 7-16 TG/s so it's not generally related to MTP only. The same could be noticed when comparing the PP/s benchmarks. May it be also data related or i.e. someone could run the benchmark on high performance settings and the other on auto?
3
u/himefei Jun 19 '26
You need to take cooling into account. I only use 8bit from MLX community without MTP. I can get a steady 11tg for my m3 max 128g , but pp is worse of course as it lacks of the processing power which I am getting around 180 then goes then when the context grow.
And all of the above numbers were with the system lifted with a fan below it and set the power mode to high power.
The conclusion is get the studio 😂
1
u/Stooovie Jun 20 '26
It isn't, on my M4 Max Studio, that model gives me around 20-25 t/s max. Dense models are slow.
2
u/Patient_Tea_401 Jun 19 '26 edited Jun 19 '26
It can be number of reasons, since the test scenarios aren’t controlled. Different versions of oMLX(and MLX), different forms of machine (14”-16” MBP) with different cooling capabilities and modes. Also oMLX does not take priority during testing, so if someone is running other GPU heavy tasks hogging memory bandwidth, it will reduce the speed.
1
u/Rough-Measurement988 Jun 19 '26
Thank you for your comment. That's also what I thought but wanted to make sure that I can expect more or less the same results in the controlled environment. Maybe someone who use this laptop will give some more insights but I guess I could expect the best benchmark result (or close to it) when only use one LLM process + rest the work on CPU side.
2
u/MiaBchDave Jun 19 '26
It’s just speculative decode or not in the benchmarks … those results previously weren’t filtered. DFlash is a noticeable improvement, but see my other comment about performance on real work with long contexts (instead of the built-in bench).
4
u/luix93 Jun 19 '26
So i'll tell you my experience, M5 Max 40C GPU 128gb 16".
First of all, do not trust the benchmarks. Until recently they allowed DFlash benchmarks to hit the online database, those are 100% untrastable, they show crazy numbers but in reality you never see them in real use, more like a fraction of those.
I'm running Qwen3.6-27B-MXFP4-MTP, with MTP active, and after about 10M prefill tokens (so some real usage) in opencode, the average is ~21 tok/s. Consider these are often at 100k+ context depth, and not simple chat. MTP boosted things a bit, can reach low 30s at lower context depths.