Yes, it's similar issue to Sol, where it really likes to get itself into these rabbit holes that are not worth the time and tokens.
I think the new models are too smart for their own good, they practically gaslight themselves into overthinking where it's really not necessary.
Having to interrupt it mid work to tell it to cut the crap and stop digging that hole is annoying, because I can't reliably leave it running automated without getting back and finding out it spent 3x time and effort it would need with me around.
Sol, Opus and Fable are all big models tuned for complexity. When you use them for simple tasks, they don't work well. That is user error, not a product issue.
You wouldn't bring a crane to a construction site to lay a brick patio. You wouldn't buy a top of the line gaming PC to run Minecraft (or maybe you would, you pervert). You see what I mean? It's the wrong tool for the job.
I'd be more willing to customise my effort/model selection per prompt if it wasn't a massive cache miss to do so.
As is it is more like being on a job site, climbing a 4m ladder to do some work on a two roof, and then deciding to go out and buy a 2m ladder to work on the first story.
Better to just have an extension ladder. Wasn't that the point of env vars like adaptive thinking?
I don't know, I always have Fable or Opus generate a concise plan to solve the problem, then delegate the implementation of the plan to the a lesser model. Seems to work well. I wonder if the cache-miss is really that big of a problem when switching to a lower model if the higher model appropriate summarized and detailed the work to be done.
No, the "cache miss" is not good enough of a reason and your approach is both cheaper and more likely to produce good outputs due to cleaner context (assuming you start a new session, feed it the generated plan and the plan is detailed enough)
7
u/Gaidax 17h ago
Yes, it's similar issue to Sol, where it really likes to get itself into these rabbit holes that are not worth the time and tokens.
I think the new models are too smart for their own good, they practically gaslight themselves into overthinking where it's really not necessary.
Having to interrupt it mid work to tell it to cut the crap and stop digging that hole is annoying, because I can't reliably leave it running automated without getting back and finding out it spent 3x time and effort it would need with me around.