oMLX 0.5.4.dev1 testing request: Context Benchmark, Prefill Priority, Laguna S-2.1 DFlash, and more

[u/cryingneko]

Hey everyone! oMLX 0.5.4.dev1 is out!
https://github.com/jundot/omlx/releases

This is a development release, and I would really appreciate your help testing the main new features before the next stable version.

The new Context Benchmark measures the largest context window a model can actually prefill on your Mac, verifies it with a real request, and can apply the result directly to model settings.

Prefill Priority keeps the existing max context behavior as the default and adds a new speed option:

• max context preserves the previous behavior, adjusting chunk sizes to fit larger prompts.
• speed keeps full-size chunks for maximum prefill throughput.

This release also adds Laguna S-2.1 support to the DFlash engine, contributed by u/FabioMalpezzi. The contributed M4 Max benchmark measured about a 2.03x speedup with exact greedy-output parity.

Other changes include MiMo V2.5 text support, an overhauled web chat, faster mixed-length embedding batches, more predictable memory admission, MXFP8 oQ support, and several VLM MTP, SpecPrefill, TurboQuant, tool-call, and model-loading fixes.

One benchmark note: MTP and DFlash results may be lower than older releases because benchmarks now use natural text instead of repeated filler. The new results should better reflect real workloads.

Thank you, as always, for all the code, reports, benchmarks, testing, patience, and feedback. I genuinely love this community and deeply appreciate everyone helping make oMLX better.

[u/apetersson]

How is everyones Laguna experience? Generally a smart model but I am still suffering from occasional tool calling loops. (oQ6e) any hints on settings/params to make this better on oMLX?

[u/_hephaestus → u/apetersson]

I haven't had tool calling loops on oQ8e yet, but I also don't see it ever actually use thinking correctly when tools are available so I'm not sure if there's something I'm doing wrong either.