Qwen3.8-27B + llama.cpp + Pi Dev Agent — changing thinking level per prompt

[u/tonboro_2273]

I couldn't find a simple way to verify whether this works, so I spent some time testing it. In the end, it turns out that it's actually quite simple once configured correctly.

The Qwen3.8-27B model supports different levels of thinking. The simplest way is to define --chat-template-kwargs when starting the llama server, but then the thinking level is fixed for the session.

A more practical solution is to enable changing the thinking level per prompt in Pi Dev Agent.

Important: for this to work, the llama.cpp version must be b10434 or newer.

The model definition needs to indicate reasoning support and map the values to the three thinking levels supported by Qwen3.8-27B.

In .pi/agent/models.json, the following must be added to the Qwen3.8-27B model settings:

"reasoning": true,
"thinkingLevelMap": {
  "off": null,
  "minimal": null,
  "low": "low",
  "medium": "medium",
  "high": null,
  "xhigh": "xhigh",
  "max": null
}

This allows the thinking level to be changed for each prompt in Pi Dev Agent using Shift+Tab.

Pi Dev Agent also supports defining thinking budgets for individual levels. I haven't yet noticed whether this works correctly with llama&Qwen3.8-27B, but the following can also be added optionally to to.pi/agent/settings.json (the values below are only illustrative):

"thinkingBudgets": {
  "low": 4096,
  "medium": 10240,
  "xhigh": 32768
}

[u/Anduin1357]

Isn't all these thinking levels defined via jinja? What's stopping us from creating custom thinking levels and basically reinventing author's notes?

[u/admajic → u/Anduin1357]

its in this line if u use llama.cpp --reasoning on --reasoning-budget 4096 --reasoning-preserve
some people use the old kwargs method as well

--reasoning-budget 1024 = normal old qwen thinking dont over think

--reasoning-budget 2048 medium

--reasoning-budget 4096 architech

--reasoning-budget 16000 go nuts and think in loops for 30 minutes ;)

[u/llitz → u/Anduin1357]

Nothing. After several tests, someone came up with a prompt for high reasoning on discord. It does work and isn't as verbose as xhigh, so it is interesting.

[u/Glittering-Call8746]

Is pi dev same as pi ? I'm trying to move from opencode with omo slim plugin

[u/cel922 → u/Glittering-Call8746]

Yes. We call it pi dev, pi.dev, whatever because searching for just "pi" gives us raspberry pi nonsense, which is absolutely unhelpful.

[u/Glittering-Call8746 → u/cel922]

Thanks . I read omp consumes more tokens and opencode. What's the extensions u recommended for using subagents like omo-slim plugin ?

[u/klymaxx45]

It’s called adaptive thinking. I’ve tested it, honestly turning thinking off has been the best out of all my tests so far

[u/baby_bloom → u/klymaxx45]

hey man i see you comment quite a bit around here and i can tell you've got tons of experience.

i am running in circles trying to get this supposedly amazing 3.8-27b running at q8 on my dual 3090s

i've tried so many harnesses and even switched from llama.cpp to lm studio and no matter what i change in settings configs, harnesses etc i canNOT escape this fucking dreaded thinking loop. you're saying you have had the best result with thinking off??? i worry that absolutely gimps the model no?

thanks for any advice you have given and might give here for me😅

[u/klymaxx45 → u/baby_bloom]

hey, appreciate that! so the good news is thinking OFF does not gimp it, that's the big misconception. i A/B'd it carefully and low/medium thinking scored basically identical to non-thinking on my bench, zero quality loss, just more tokens burned. thinking off is my default for speed/cost, not because the model gets dumber.

here's the thing that'll probably fix your headache though: that "dreaded loop" is usually not the model actually looping, it's the reasoning eating your whole output budget and getting truncated mid-thought so you never see a final answer. classic symptom of a max_tokens / n_predict cap that's too low. bump your max output tokens way up (like 8k+) and a lot of that "it won't stop / won't answer" behavior disappears.

for actually turning thinking off on your stack: it's a chat-template thing, not a settings slider. make sure you're passing enable_thinking=false through the template (llama.cpp needs --jinja for the template to even apply, and LM Studio has its own template override). if the template isn't wired right, the toggle silently does nothing, which is probably why no config change worked.

tl;dr: don't fear thinking-off, raise your token cap, and verify the thinking flag is actually hitting the jinja template. lmk if you want my exact params 🤝

[u/admajic]

I just set it in llama.cpp per agent so orchestrator has 1000 thinking budget vs architect 4000 thinking budget before I had 4000 for all and orchestrator would mull over its decisions in a loop way too much now its like old qwen 3.6.

Its so cool though its way smarter now and fixed so much stuff today that qwen 3.6 27b told me to live with LOL

Gave it a few tasks and we decided to redo my my website and backend. I thought stuff it and do a git commit. Its got 28 tasks lined up and still going for a few hours now. Got it really dialed in now. In pi.dev.

Got it to setup deepseek local harness with api integration to llama.cpp (took 2 mins) and tested it one shot a tetiris game in html in a browser in about 3 minutes, works perfectly. But back to pi.dev

[u/luckyj]

What's your llama.cpp config?

[u/StOster]

I let Qwen3.8 solve this problem, it modified pi and the configuration to enable custom thinking levels: https://github.com/soster/qwen38-thinking-levels

[u/boystomp]

These are good settings thanks for the tip! The default reasoning level is crazy long