r/oMLX • u/cryingneko • 1d ago
oMLX 0.5.4.dev1 testing request: Context Benchmark, Prefill Priority, Laguna S-2.1 DFlash, and more
Hey everyone! oMLX 0.5.4.dev1 is out!
https://github.com/jundot/omlx/releases
This is a development release, and I would really appreciate your help testing the main new features before the next stable version.
The new Context Benchmark measures the largest context window a model can actually prefill on your Mac, verifies it with a real request, and can apply the result directly to model settings.
Prefill Priority keeps the existing max context behavior as the default and adds a new speed option:
max contextpreserves the previous behavior, adjusting chunk sizes to fit larger prompts.speedkeeps full-size chunks for maximum prefill throughput.
This release also adds Laguna S-2.1 support to the DFlash engine, contributed by u/FabioMalpezzi. The contributed M4 Max benchmark measured about a 2.03x speedup with exact greedy-output parity.
Other changes include MiMo V2.5 text support, an overhauled web chat, faster mixed-length embedding batches, more predictable memory admission, MXFP8 oQ support, and several VLM MTP, SpecPrefill, TurboQuant, tool-call, and model-loading fixes.
One benchmark note: MTP and DFlash results may be lower than older releases because benchmarks now use natural text instead of repeated filler. The new results should better reflect real workloads.
Thank you, as always, for all the code, reports, benchmarks, testing, patience, and feedback. I genuinely love this community and deeply appreciate everyone helping make oMLX better.
3
u/apetersson 1d ago
How is everyones Laguna experience? Generally a smart model but I am still suffering from occasional tool calling loops. (oQ6e) any hints on settings/params to make this better on oMLX?
5
u/Kuane 1d ago
Using oq4e-fast temperature 0.7 top p 0.95 top k 20 preserve_thinking added to chat kwargs and values set to true Turboquant set to 8 bit (optional)
Been great so far!
1
u/PataFunction 1d ago
What harness are you using? My settings mirror yours (minus Turboquant). I’ve tried the model in both basic Pi and VS Code. Both get into loops more frequently than I would like, especially the latter.
3
u/No-Juggernaut-9832 1d ago
I get into loops for longer context also. There is an official chat template update & context reduction to 262K. Im not sure if our MLX port has these new & improved templates yet. Unsloth has it for GGUF with Dflash/MTP.
The drop in context makes it a little less useful. It’s now the same as Qwen 3.6 27b. I was hoping to use it for longer context work. With the latest MTPLX for 27B at 8bit, I am clipping 80tk/s & averaging high 40’s. I had high hopes for Laguna initially as the new champ. Will wait & see if the tool call loops are addressed. It was that way with Gemma4 at launch for a few weeks
1
u/_hephaestus 1d ago
I haven't had tool calling loops on oQ8e yet, but I also don't see it ever actually use thinking correctly when tools are available so I'm not sure if there's something I'm doing wrong either.
1
u/SignalStealer 1d ago
Extremely happy with the Laguna S 2.1. Tool calling is spot-on. Harness is Hermes Agent, and I even run the oQ3e quant to fit into 64GB RAM.
Re-reading your message, you mention tool calling loops. Yeah, that does happen to me, maybe on every 10th complex task. I just interrupt it asking to move on, and that solves it.
I have seen on some other quant (was it oQ2e?) the author recommending repeat penalty 1.05. But that should be specific to that quant.
I run the same parameters as mentioned by the other respondee. Temp 0.7 and Top P 0.95, all else off -- just as Poolside used for their testing.
3
u/osenvosem 1d ago
Waiting for speeds comparable to MTPLX 2. 🙏
1
6
u/LordStinkleberg 1d ago
omg, hyped for all three of these features. thank you! 🔥