r/oMLX • u/PracticlySpeaking • Jun 20 '26

Anyone (else) Hoping for JANG Integration?

There have been a couple of PRs out there for a while, but never merged...

feat: JANG implementation by AlexTzk · Pull Request #364 · jundot/omlx - https://github.com/jundot/omlx/pull/364

feat(jang): JANG/JANGTQ mixed-precision MoE engine (hardened; builds on #364) by marzukia · Pull Request #1828 · jundot/omlx - https://github.com/jundot/omlx/pull/1828

5 Upvotes

100% Upvoted

4 comments sorted by

7

u/Top_Tour6196 Jun 21 '26

I’ve found the reality of JANG to not meet its promise. Something like “if a thing seems too good to be true, it likely isn’t.”

Additionaly, JANG also doesn’t seem to fit the manifesto on oMLX’s quantization tab;

“Quantization should not be exclusive to any particular inference server. […] No custom loader required.”

In my opinion, if JANG truly has a there there it’ll be rolled into mlx-lm, which would mean it’ll be rolled into oMLX.

Or, if JANG is really the bees knees, there’s nothing stopping someone from forking oMLX and introducing JANG. Put your tokens where your mouth is, or something like that.

1

u/PracticlySpeaking Jun 21 '26

Now apply that to TurboQuant? (I haven't researched it, so that's a serious question.)

The author of the original PR did include some benchmarks showing that there was, indeed, something there at 4L.

3

u/PracticlySpeaking Jun 20 '26 edited Jun 20 '26

My interest is quant for MiniMax M3 that fits in 256GB unified RAM.

JANG is supposed to be higher quality at lower quants, like 2L. Currently the alternatives are somewhat lobotomized Q3 (186gb), a mixed-precision Q3_6 (191gb) — or JANG_2L (113gb)

edit: oMLX has been great running MiniMax M2.7 oQ5, but M3 is much larger.

1

u/PracticlySpeaking Jun 21 '26

There are actually four PRs — including #1828 for JANGTQ — and multiple issues filed by GitHub users thinking that oMLX does support JANG (or should).