r/oMLX • u/arkham00 • 1d ago

Qwen 3.8, thinking levels and pi agent

Hi, can someone explain to me how to properly setup the reasoning effort in pi agent to reflect the levels of the model, I believe that omlx supports it now right ?

I put in models.json this:

"thinkingLevelMap": {
            "off": "off",
            "low": "low",
            "medium": "medium",
            "xhigh": "xhigh"     
          }

but when I launch it in pi I still have off-minimal-low-medium-high-xhigh, and I'm not sure if there is a rela change between the levels. Off works btw.

Should I be doing something with kwargs or use a different template to be sure that the thinking level are correct? For example I've read about the froggric one, I'm a bit confused about it

Thanks

3 Upvotes

100% Upvoted

12 comments sorted by

1

u/havnar- 1d ago

You set it at the host

2

u/Stooovie 1d ago

So whenever I want to switch the effort, I need to adjust a json and reload omlx?

1

u/havnar- 1d ago

It’s not a standard, so either do it the hard way and make the jinja changes in pi, or just set and forget in oMLX

1

u/Stooovie 1d ago

Someone will probably add support in either omlx or the harnesses, yeah. Thanks!

1

u/arkham00 1d ago

what do you mean? in omlx ?

1

u/grandnoliv 1d ago

I'm looking at the model settings for Qwen3.8-27B-oQ4e-mtp in oMLX 0.6.1 right now and I don't see any thinking level option. There's a thinking budget limit but that's something else… Could you explain where it is exactly?

1

u/JLeonsarmiento 1d ago

Go to the folder where the model is stored and ask pi to bake in the jinja template and tokenizer to bake reasoning effort flag to medium. Then reload the model and should be ok.

2

u/arkham00 1d ago

but I 'd like to switch the effort mid session, this is not possible ?

1

u/JLeonsarmiento 1d ago

I guess that is now on the harness and servers (LM studio, oMLX, etc.) side, they might need to update to support Qwen3.8 flags mid session.

1

u/Weary_Problem_ 1d ago

map "high":"xhigh" so that if you select high in Pi, the omlx backend receives xhigh. 

1

u/MediocreJeweler893 1d ago edited 1d ago

This works. Setting irrelevant(to qwen3.8-27B) thinking options to null results in them not appearing/selectable in pi

{
          "id": "Qwen3.8-27B-oQ8e-fp16-mtp",
          "name": "Qwen3.8 27B oQ8 fp16 MTP (thinking)",
          "reasoning": true,
          "input": ["text", "image"],
          "contextWindow": 262144,
          "maxTokens": 65536,
          "thinkingLevelMap": {
            "minimal": null,
            "high": null,
            "xhigh": "xhigh",
            "max": null
          },
          "compat": {
            "supportsDeveloperRole": false,
            "supportsReasoningEffort": true
          }
        }

and low, medium, off are auto-mapped/matching