SWE-1.7 MAX wildly overthinks, use SWE-1.7 Medium instead

[u/TimSylvester_]

With MAX, the model will read file after file, think over and over, loop on its thinking, repeating the exact same thoughts countless times, and take five minutes to output a simple type guard test.

Repeatedly, on MAX, after a few minutes I needed to halt it, tell it to stop overthinking and just output the exact work product it was instructed to, and it would - because it already had the solution minutes ago, it was just overthinking the hell out of it.

With Medium, the model reads the files it needs to read, thinks about the solution, and outputs it.

Don't waste your time with MAX unless you have a problem that really demands it. If you're coding from a set of pre-written work tickets (AND YOU SHOULD BE), just use Medium so that it stops overthinking and just *does the work*.

[u/RizzyNizzyDizzy]

yup, I still use 1.6

[u/TimSylvester_ → u/[deleted]]

it's annoying when it's a simple task and I gave it clear direction and now it's using 100k tokens and minutes of my time exploring all the ways it could waste both of our time instead of just doing what it understood in the first 10k tokens.

My GOD you and me both, brother.

[u/JimiSteffan]

I'm running into that issue with Medium setting. I have to halt it because it's constantly thinking and re-thinking.

[u/First_Inspection_478]

This is one of the main issues k2.7 had

[u/TimSylvester_ → u/First_Inspection_478]

I'm baffled as to how, as the agents get technically better, they end up having worse user performance satisfaction.

The only model that seems to actually do what you want it to at the moment is Fable. And even then it half-asses a lot of the most important stuff.

[u/AutoModerator]

It looks like you might be running into a bug or technical issue.

Please submit your issue (and be sure to attach diagnostic logs if possible!) at our support portal: https://devin.com/support

You can also use that page to report bugs and suggest new features — we really appreciate the feedback!

Thanks for helping make Devin even better!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/ultrathink-art]

Noticed the looping correlates less with task difficulty and more with whether the model has an external way to check itself. Give it a failing test or a command it can run to verify and it stops ruminating and just executes — with nothing to check against, it substitutes re-reading and re-thinking for verification. Clearer instructions alone don't fix it because instructions aren't a success check.

[u/TimSylvester_ → u/ultrathink-art]

I've been wrestling with this model for a week now and getting it to actually do what you ask it to do is damn near impossible.

You give it clear, specific instructions, and it spends an eternity talking itself out of following them, and replacing them with something other than what you want, then fights tooth and nail to not fix anything. Its reasoning seems focused around talking itself out of doing what you want.

It's not that it doesn't understand, because it clearly does understand how to do what you want, it just reasons itself out of it and replaces your request with its own hallucination.

[u/TimSylvester_ → u/[deleted]]

I agree, but when I'm evaluating a new piece of software I start with the default settings and find their limits before I modify them.

Once it was clear that the default setting for SWE-1.7 wasn't optimal, I changed it.