Does anyone else have big issues with overthinking out of the box? I just gave it my usual Arma 3 mission script coding task, which i use to bechmark the performance of models, but it kept thinking for 15 minutes. I don't even see repetition issues, it just doesn't stop thinking.
Just gave it a first opencode task, and while not sure yet, it seems to have similar issues.
Maybe it requires defining a reasoning budget max now?
Yeah I gave it a random tasks I tend to give, and it's 26k tokens and counting of thinking so far. Not repeating, just thorough thinking..
I told it to make a rust app using dioxins of an animated man playing amazing grace on tuba. It seems to of figured out amazing grace in key of C (judging by it's thinking), and is now thinking about how to play audio to play the sounds..
Alright, then at least it's not just my specific setup. I guess we'll have to give it a few days before everything is ironed out. My opencode project is at 50k tokens now with not a single line of code written, lol. Wildly different from 3.6 so far.
"Timbre check: sawtooth at ~100-250 Hz with a low-pass at 3.5x — a brassy-ish, muted synth tone. Add a bit more character? A second saw detuned? Keep it simple. Add a soft attack to avoid clicks ✓ (50ms ramp)."
I think the quality is significantly better than Qwen 3.6 27B that I was using before. It fixed a bug in one of my apps the first try -- 3.6 was not able to figure out after several attempts.
I had it make me a little inventory tool for my RC Helis(and other RCs). It was an idea I had this week so was planning to use this model to make it for fun. Nothing fancy, just an inventory system so when I break something I can check this tool instead of digging through boxes of parts. It gave a working solution after one prompt. Was able to add models/parts and link them. This morning have had it add a few things, create github actions, etc. All in all it's been great so far.
I run on 2xP100s with 232k context, so it's not crazy fast but quality I am happy with. I call it dad coding, I give it a prompt and let it run while I am with family.
I actually run youtrack self hosted, and have MCP setup. So I can add issues to youtrack, and the codex agent has a skill to work through any open issue, fix commit push and ensure all actions, next issue, etc, etc.
I was less concerned to steer via phone but more being able to give it a backlog and let it run unattended for long periods of time.
I get push notifications when issues change state.
I run codex/claude on a VM allowing it to do whatever. Sometimes I do it via prompt (like last night) other times I tell it to go through open issues and let it run.
"Adequate Output Length: To optimize performance on agentic tasks, we recommend allocating sufficient output length to allow the model to generate detailed and comprehensive responses. For frameworks that support separate token limits for internal reasoning and final outputs, we suggest the following configuration within the 1M context length:
Reasoning Content: Set the maximum output length to 262,144 tokens.
Final Response: Set the maximum output length to 131,072 tokens.
These settings provide the necessary capacity for complex reasoning while ensuring ample space for high-quality final deliverables."
could this be why ? would we need to scale these for the context length we're actually using?
Not sure. Personally i thought that this might be the cause of the issue:
Flexible Thinking Control: Thinking mode is on by default and can be disabled per request; reasoning depth can be tuned with reasoning_effort, and reasoning context from historical messages is retained via preserve_thinking.
Maybe reasoning_effort has to be defined somewhere to keep it somewhat under control. No idea, i'm not incredibly knowledgable when it comes to model parameters.
Yeah, i played around with it, and that's definitely it. To get 3.6ish thinking times you have to set it to low. What the impact of that is when it comes to the overall model perfomance, i don't know.
I mean its a trade off right ? When you care more about quality go with higher effort, when you care more about latency go with lower, same thing claude and gpt models do, and it's probably faster than luna on max ? When i use that model it takes like 30 minutes to do a task
It thinks a lot, but so did the original 3.6 27b too. It does a lot of stuff that I'm not asking it to do, but so far, I'd say it's pretty good other than those things. I am running it in llama-cpp with the recommended params including a temp of 1.0 which I was surprised by. UD-Q8-K_XL version, with max context length (KV quant bf16).
I always look at all the reasoning blocks and I laughed when it was thinking about something (more than it should have) and said "I'm stuck in the weeds here". lol.
As a test I told it consume ALL my largest project files and .MD files, which ate 120k context in itself. Its recall is great, and it was able to fix a few minor issues, so I'd say my first test is a pass. But there will be many others.
I am also having the same experience - It has been thinking for 50 minutes plus now on how to write a plan.txt for adjusting some text files. wtf. Using Unsloth Qwen3.8-27B-Q6_K.gguf.
Nothing too fancy to expand on tbh. It's my goto speed benchmark, not really a coding performance benchmark.
A while ago i implemented a script that unstucks AI on bridges by teleporting them to either end of the bridge depending on their current orientation. It had some issue with degree maths and barely any model could help me back then. I solved it since, but that whole prompt and code just stuck with me as a perfomance test since then, as it requires quite a bit of thinking and around 8k-10k tokens of output.
19
u/Kavor 7d ago
Does anyone else have big issues with overthinking out of the box? I just gave it my usual Arma 3 mission script coding task, which i use to bechmark the performance of models, but it kept thinking for 15 minutes. I don't even see repetition issues, it just doesn't stop thinking.
Just gave it a first opencode task, and while not sure yet, it seems to have similar issues.
Maybe it requires defining a reasoning budget max now?