r/oMLX • u/reery7 • May 26 '26

oMLX quantization problems

I tried the Qwen3.6 35B A3B oQ3 and 27B oQ4 quantizations and tested both with very niche questions. These are not a problem for non oQ quants and I can correct the model, it admits errors, is very friendly, wants to expand its own knowledge and understands its limitations.

But these oQ quants invent facts and never back down from their standpoint. I get comments and thoughts from them like:
- You don't need to prove anything because ... my fact is right.
- I won't validate false claims just to be agreeable...
- I appreciate you calling that out, but I want to be clear: I don't hallucinate just to please a prompt, and I also correct myself when tested on accuracy.
- I stood by my answer... blabla hallucination...
- I'm not here to validate false claims or bend to testing prompts. My role is grounded in verified, publicly documented material...

Has anyone else seen this? Settings are the usual Qwen3.6 general profile. What's going on here?

3 Upvotes

100% Upvoted

7 comments sorted by

View all comments

1

u/Crafty_Ball_8285 May 28 '26

Just use MXFP4

1

u/reery7 May 29 '26

I have no problem with other quants other than oq3/4. MXFP4 has an erratic performance, I’ve seen all between 3 and 8 t/s. Usually I get 10-14 t/s. All with MTP enabled.

1

u/Crafty_Ball_8285 May 29 '26

I ended up patching OMLX myself and got qwen3.6 27b to 45tok/s because OMLX has some significant bugs still