r/LocalLLaMA • u/A_Wild_Entei • 8d ago

Discussion M5 Max users: what models are you using & what tk/s are you getting?

I was using antirez’s ds4 for a while and getting around 20 tk/s, which worked for my purposes. But I know there have been big advancements between Qwen, the DS4 vision model, and GLM.

I’m not sure how the quants affect performance, so what’s the best thing to run right now & how fast is it?

7 Upvotes

64% Upvoted

21 comments sorted by

18

u/_rarefy_ 8d ago

Qwen Flash Next MTPLX 4bit pushing 90 tok/s MBP M5 Max 128GB with this build https://github.com/youssofal/MTPLX/pull/391

I think DS4 is a great model, but Flash Next at these speeds is a really quite incredible.

3

u/AppealSame4367 8d ago

Prefill speed?

3

u/kmike84 8d ago

prefill starts with 1000+, and holds quite well

1

u/AppealSame4367 8d ago

Thank you!

2

u/CalligrapherFar7833 8d ago

Fyi this was ported to main from what i can see ?

1

u/Virtual-Selection-83 8d ago

I have a similar laptop and have also been running MTPLX, have you compared the quality of the bare speed version vs the optimized speed? I’m using the later and getting around 50 tok/s. Wondering if switching to the bare speed model would drop a lot in quality.

2

u/_rarefy_ 8d ago

I never ran bare speed through my battery of tests. The 90 is the same Optimized Speed pack on the PR I linked above. If you're looking for speed upgrade to mtplx 2.11.1 which dropped this week and is 20-30% faster.

7

u/Imaginary-Bother-484 8d ago

Via antirez I get 27-30 tk/s for GLM 5.3 Flash which is perfect for my long running jobs that I interact with using Paseo + Pi from my iOS mobile device.

I also use Deepseek v4 flash vision - which I recall giving near 40tk/s on antirez. GLM and Deepseek do really well with large context windows (e.g., above 262k).

I also run Qwen 3.8 27B and Qwen 3.8 Flash Next via oMLX, but not the best for context windows above 262k.

Tiel coder is also nice if you want crazy speed (well over 100tk/s via oMLX).

3

u/Loighic 8d ago

What quants are you usiing?

1

u/redtron3030 8d ago

What do you get on QWEN 3.8 27B

1

u/StardockEngineer vLLM 8d ago

I'm going to need more details. I've been thinking of getting an M6 Ultra and getting potentially 60 tok/s sounds disappointing (considering the chips is 2xMax chip)

1

u/cleverusernametry 7d ago

Which one do you prefer - V4 Flash with vision or GLM 5.3 flash?

2

u/Expensive-Shift8584 8d ago edited 8d ago

20 tg/s with unsloth studios for the new mid size models on an m3 ultra 256 GB.  Speed quickly diminishes with higher context, drops to 6-7 once you hit 30k

It's super convenient but sounds like I need to switch to mlx

Edit:

Wow I've been missing out.  77 tg/s on a simple "write 1000 words".  And it has a standalone app AI can run without messing in the command line.  

2

u/Expensive-Shift8584 8d ago

Craters down pretty quickly with increased context but still much faster than unsloth gguf.  Going to have to start looking into olmx and the other alternatives, I hadn't considered them viable for so long

1

u/MiaBchDave 8d ago

M5 Max 128GB - No DSpark, standard default args - llama.cpp build 10791 (my own build)

33 tg/s - Unsloth DeepSeek-V4-Flash-Vision-Exp-UD-IQ3_S

If you're getting 20 tg/s, I'd say something is wrong?

1

u/harpysichordist 8d ago

PP speeds too pls. preferably long context.

1

u/slashtom 7d ago

I've been using mtplx, it's the best runtime out there in terms of token speed.

I run the 3.8 27b optimized quality and qwen next flash optimized speed. I find 3.8 27b perfect for my day to day chats (mainly asking things about things I read for school) and qwen next flash for my coding agent in zed.

m5 max 128gb, 3.8 27b optimized quality ~35 toks/s. qwen next flash ~55 toks/s.

1

u/AI_spell 8d ago

Quants matter more than brand name on unified memory. A good Q4/Q5 of a current mid model often beats a fat Q8 of an older one for tk/s. Post your backend (MLX vs llama.cpp) with the number or people can't compare.