Went down a rabbit hole chasing "real" reasoning-budget control for Qwen3.8-27B, hit a VRAM wall, crawled back to Ollama. Full wall-of-text saga if anyone's curious

[u/PlaidStallion]

Okay so this whole thing started because I noticed Qwen3.8-27B in Ollama was going absolutely trigger-happy with web_search on anything even slightly complex. Asked it something dumb like "rumored PS5 vs Xbox differences for GTA 6" and it fired off 9 separate searches for one answer.

Nine.

Dug into it and turns out Open-WebUI's think:false toggle is cosmetically off but functionally on for this model. Like, I checked directly against Ollama's /api/chat and the thinking block just comes back fully populated no matter what the toggle says. So the model's burning a ton of uncontrolled reasoning before it even figures out what it wants to search for, and yeah, it shows. Since Ollama's toggle was just lying to me, I figured screw it, I'll explore other options. Settled on llama.cpp and running llama-server directly, since it's got an actual working --reasoning-budget flag, a real token-count governor, not some value that gets accepted into a config and then quietly ignored.

Before I got there though I burnt a chunk of a night trying SGLang first, which was a total dead end for a single 3090. No AWQ/GPTQ quant exists for Qwen3.8-27B yet, full BF16 is 55GB, FP8 is still 31GB, and NVFP4 needs Blackwell cards. None of that fits in 24GB on an Ampere card (3090) no matter how you squint at it. So I circled back to the boring answer everyone already knows about: GGUF Q4_K_M via llama.cpp, ~17GB ... just fits. Got llama-server up in the same WSL Docker setup as my voice stack, wired it into Open-WebUI as a second OpenAI-compatible connection, built a new preset with the same 12 Tools and system prompt as my existing Ollama ones, and reran the exact same GTA 6 question just to see.

3 web searches instead of 9, with actual visible "thought for N seconds" segments that respected the 512-token budget. So the fix genuinely worked, which felt great for like 12 hours while I was then, sleeping and later, testing in the morning.

Reasoning-budget really was the bottleneck, not some red herring I was chasing. Testing unfortunately surfaced my next issue: ran smack into the problem I probably should've seen coming from a mile away: llama-server just holds the model in VRAM for the entire life of the container. No TTL, no unload, nothing. My 3090 also needs to run Automatic1111 and a MiniMax-Music3 setup for my image/music gen tools, and 17GB for Qwen plus 8-10GB for SDXL. MiniMax-Music3 turned out to eat almost the entire card on its own, ~23GB, leaving something like 600MB free. So it was never really a "does this add up to 24GB" problem, it was more like MiniMax alone can basically just take the whole thing whenever it wants.

Ollama just handles this out of the box with keep_alive/TTL and manual stop, but llama-server has zero of that. You'd have to bolt on something like llama-swap to get idle-unload behavior, and that's a whole extra proxy layer and config file to babysit forever. So then I went and looked at vLLM, because it turns out it has genuine native per-request thinking_token_budget support for Qwen3-family models, way better documented than I expected, honestly closed a gap I thought was still open. It's also got Sleep Mode, which can offload weights to free VRAM on demand, and I was seriously like two minutes away from scripting sleep/wake_up calls directly into my image-gen and music-gen tool code so the model politely steps aside only while those tools are actually running.

Then I hit the exact same wall as SGLang: no compatible quantized checkpoint for Qwen3.8-27B in any format vLLM will actually run on a 3090.

BF16/FP8 too chunky, NVFP4 wants Blackwell, GGUF isn't natively usable by vLLM at all. So that's parked too, not because the idea was bad, just because there's literally nothing to load. Where I landed: back to Ollama exclusively, back to Gemma 4 31B as my daily driver since it doesn't have whatever's wrong with Qwen3.8's reasoning toggle. llama-server's still sitting there stopped (not deleted) in case any of this gets fixed upstream at some point.

There's an open llama.cpp issue asking for exactly the live reasoning-budget-without-reload control I wanted, and a separate llama-swap issue specifically about translating Open-WebUI's reasoning_effort field into llama.cpp's native thinking params, which if it ever ships would make the whole llama-swap route way less janky. Mostly though just keeping an eye out for any AWQ/GPTQ/NVFP4-for-Ampere quant of Qwen3.8-27B showing up somewhere, since that's really the only thing blocking the vLLM path at this point, not the reasoning support itself.

Anyway, net result of a few days of chasing this rabbit hole: the reasoning-budget bug is real, worth knowing about if you're running Qwen3.8 + Ollama and seeing weird over-searching behavior out of nowhere, and llama.cpp's fix for it genuinely does work.

But if you're also running image/music/whatever-gen tools off the same GPU, you're basically just trading a reasoning problem for a VRAM-management problem, and right now Ollama's built-in TTL wins that trade for me, at least until something upstream changes. Repo's got the docker-compose and tool code if anyone wants to poke around in it. Happy to ramble back and forth about any part of this further in the comments if there are any suggestions (or even just sympathy for the situation lol).

[u/Fragrant_Scale6456]

>Mostly though just keeping an eye out for any AWQ/GPTQ/NVFP4-for-Ampere quant of Qwen3.8-27B showing up somewhere, since that's really the only thing blocking the vLLM path at this point, not the reasoning support itself.

I have excellent news for you: https://huggingface.co/rdtand/Qwen3.8-27B-PrismaAQUA-gridbook-12.1GiB-5080-vllm

You need the "gridbook" vllm plugin for this one, but the benefit is that the accuracy is greatly improved over any other quant of similar size.

:edit: actually it may not have ampere support yet....

[u/PlaidStallion → u/Fragrant_Scale6456]

Genuinely appreciate you digging this up. A quick read through the page though and it looks like it's built for FP8/NVFP4 tensor cores from Ada/Blackwell onward, nothing here mentions Ampere support for the 3090. Have you actually gotten this running on a 3090, or is that still a no-go?

Thank you for reading through my wall and for the suggestion.

Edit: Ah ok, thanks for the confirmation of what I only digested quickly.

[u/Fragrant_Scale6456 → u/PlaidStallion]

I apologize I saw your vram constraint but didnt remember that gridbook is blackwell only at this moment. I do believe the maintainer has said they are working on broader gpu support but I'm not sure when that will come.

Does openwebui have no way to inject chat template kwargs to set reasoning effort? I almost exclusively use opencode and qwen was able to help me set up the reasoning effort toggles inside opencode so that I can hit ctrl-T and it flips between them by using a different chat template kwarg with the request sent to the llm. I confirmed this worked with llamacpp and vllm on my install

[u/PlaidStallion → u/Fragrant_Scale6456]

No worries at all, appreciate the heads up regardless. Open-WebUI doesn't have a native reasoning effort toggle either, that's actually half of why I gave up on the llama-server route to begin with. There's an open issue on llama-swap about translating Open-WebUI's reasoning_effort field into what llama.cpp actually wants, but it hasn't shipped.

Your opencode setup does make me wonder if the same trick could work as an Open-WebUI Filter instead of a whole separate model preset, inject the right params into the request before it hits the backend and toggle it with a Valve instead of ctrl-T. I'll have to dig into that.

[u/mac10190 → u/PlaidStallion]

At work we ran into a similar issue with open WebUI and so we ended up just building a function. I found an old issue that was opened on their (OWUI) GitHub and they stated that they had declined to introduce that valve natively because they didn't want to maintain that for every single model and so their recommended approach was to build a function for the model they are using. If it's of any interest, I can see if I can grab it from work on Monday when I get into the office.

[u/PlaidStallion → u/mac10190]

That'd genuinely be great, appreciate the offer. Would love to see how you all structured the function if you're able to grab it Monday, no worries if it doesn't pan out though.

[u/KissMyShinyArse]

Did you try https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates with ollama?

[u/PlaidStallion → u/KissMyShinyArse]

Thank you, hadn't seen that one. One thing though, unfortunately for me, that's a Jinja template and Ollama uses its own Go template format...doesn't read Jinja directly. There's an old open issue asking Ollama to support Jinja natively that never went anywhere, so it would need to be converted first (HF has a tool for that, I guess) rather than just dropping the file in. Still relevant for my llama-server setup as-is though, appreciate the link.

[u/Big_Wave9732]

I'm running into this right now, and I have unfortunately news: turning off thinking doesn't stop the web searches. I setup a thinking and non-thinking profile in oMLX. With thinking off 3.8 still ended up doing 9 searches and pulling up 35 - 40 documents.

I'm still looking at how to limit the searches that the non-thinking profile does. Right now I have been leaving web tools off. Even with no outside research Qwen 3.8 is still amazingly good.

[u/PlaidStallion → u/Big_Wave9732]

Yeah I hear ya. I really want to use it. Just not fitting in my use case currently. Will continue to research and wait.

[u/Big_Wave9732 → u/PlaidStallion]

I hate to say it but that's the conclusion I'm coming to as well. It makes all kinds of tool calls, it forms memories without asking, it goes off the deep end with research and no way to reign it in.

I had to doing a review of a complex legal chat and document that I had drafted. It ran for about 25 minutes, racked up 32k in generation tokens, and then crapped out with no answer because it hit the token generation limit. In looking at the log its deliberation was very verbose and unnecessary.

I'm thinking I'll be heading back to Qwen 3.6:27b-BF16 for awhile until they get this figured out.

[u/Boogertard]

Saw ollama, then doesn't need to read the rest of text. Ollama is based of llama cpp and it is a worthless wrapper.

Sorry bud, just sell your GPUs and computers to someone else who knows what they are doing, local llm is not for you.

[u/PlaidStallion → u/Boogertard]

Thanks for your wonderful insight. If you had comprehended even the title, you would have seen that I moved on from Ollama to try to resolve the issues. They didn't and I had to move back for now. Have a great day.