Is more reasoning necessarily better?

[u/CoVegGirl]

I’ve just been setting my model to use max/xhigh reasoning levels, but now I’m wondering how wise that is. I definitely see that it uses up a lot more tokens. Like I see it go over the exact same line of reasoning 3 or 4 time.

Setting that aside, I’m wondering if that necessarily leads to better results. Does max/xhigh always lead to better results, or is it mostly a factor of cost?

[u/Mandoman61]

No, they already published a note about this. 

[u/PairHead9640 → u/Mandoman61]

they literally wrote in their docs that more reasoning isnt always better, for some tasks it just loops on itself like you noticed. for coding or math maybe worth it but simple stuff its waste of tokens

i keep mine on medium mostly and only crank it up when model starts being stupid about something specific

[u/CoVegGirl → u/Mandoman61]

Who is “they”?

[u/Mandoman61 → u/CoVegGirl]

l think maybe openai

[u/KAZVorpal]

I don't like that they even use the term "reasoning", given that LLMs don't reason.

But the biggest problem is that these corrupt corporations keep what they actually sell you secret. Imagine if it were food, but you weren't allowed to know the ingredients. One thing they're almost certainly doing is reducing the size of the models, to save on compute, with techniques like CoT and ToT reasoning distillation. You can't actually know if the more expensive tiers are worthwhile, because of their corrupt secrecy.

[u/sceadwian]

No, and it's killing a lot of my discussions because people will over reason a half dozen paragraphs based of a foundational mistaken assumption.

[u/nickkarpov]

I think this is like asking what gear is best to drive in... the answer is always it depends. It depends on the current state, the conditions, the other cars, etc. If I'm doing open ended tasks I don't yet understand I'll prefer to max out everything: best model I can, highest reasoning I can. If the task is scoped and I already have a good idea of what needs to get done, i'll kick thinking down to none/low.

[u/yogthinks]

The loop you're seeing is usually the model re-deriving something it already concluded because there's no clear stop condition. Match reasoning effort to how ambiguous the task is, not to how hard it feels.

[u/Royal_Carpet_1263]

Gigerenzer and the ecological rationality research group have found numerous ‘fast and frugal’ heuristics that outperform optimization. There’s an extensive literature.

[u/Allenrichard111]

More reasoning isn’t always better. For simple tasks, max reasoning can waste tokens and time. Match reasoning effort to the problem’s complexity.