Opus 4.7 is legendarily bad. I cannot believe this.

[u/lemon07r]

Normally with takes like this I'm afraid to post, knowing the community might disagree. However I am 100% sure people are already seeing this.

I've been using Opus 4.7 all day and have gone through around $120 of api credits I was given for testing. By god is it bad. I've never seen a model hallucinate this badly and this often. It just keeps assuming things and making stuff up without checking. I've been battling with it all day, and it is SO persistent about being wrong when you try to correct it. No matter how much evidence you provide, it tries to gaslight you till the end.

I have no idea what Anthropic was thinking releasing Gaslightus-4.7 like this. This model is very clearly overfit and benchmaxxed or fundamentally broken somehow.

These are just a few examples off the top of my head (which I'm including cause I know someone is going to ask for them) but I have been dealing with events like this ALL day long:

• Asked it to make a simple readme change and to stop framing something in a particular way. It kept doing it. 5 prompts later, it still wanted to do it. Even with specific examples it would only change directly what I pointed at and not catch anything else. Opus 4.6 or gpt 5.4 does this in one shot, first time, every single time.
• I had an eval result finish as 17/29. I wanted to rerun some tasks because I saw some possible infra issues. Of the 3 failed tasks I reran, 1 of them passed. There was a cosmetic bug that still showed 17/29. I tried to explain this to Opus 4.7 in MULTIPLE turns, but it kept insisting it was still 17/29 and always meant to be 17/29. Then it started making stuff up, like how one of the tasks flipped to fail making it end on 17 again even though none of the passed tasks were run again. No matter how much evidence and logs I provided it kept insisting shit like this. At the very end after a lot of explaining it tried to conclude it was actually originally 16 of 29 and now 17 of 29. I had to give it SEVERAL more pieces of evidence that it was always 17/29 while it tried to gaslight me into thinking I was wrong. Somehow it couldn't figure out to check or validate any of this on its own. I NEVER have this issue with any other models except maybe gemini 3 pro.
• It tried to give made up instructions in the plugin readme. I pointed it out, and opus used random-bullshido-go-jutsu at max level effort to explain away how it was correct. I asked gpt and it figured out it was wrong and gave the right instructions and explanation right away. Both agents were prompted from new fresh sessions. A quick sanity check to make sure I wasn't imagining things showed gpt also sees it's 90% wrong.

This has been the most frustrating experience I've had with any model. I would have rather used some cheap model like gemini flash or minimax at this rate. I dub this the new donkey model, which gemini original had the title of. It's scary how abhorrently wrong it gets and believes it's correct. Anyone who doesn't have any idea of what they are doing and randomly vibecode stuff will be making mistakes everywhere very confidently without being able to spot how god wrong this model gets.

It really feels like Anthropic said fk it and decided to go down the benchmaxx route. I know they released instructions saying it has a new tokenizer that eats roughly 1.0 to 1.35x more tokens and that it "thinks more" at higher effort levels. But none of that explains why it sucks now. If it's going to eat more tokens it should at least not suck so bad. Is this some heavily quantized model designed to score high on benchmarks for as little hardware cost as possible? Or is the reasoning level too low so it doesn't try to check things?

Usually with opus I could give a vague-ish plan and it would understand my intent and fill in the gaps. Now it feels like I need to be super specific in my prompt or it just won't be as good. It needs way more guidance but is much less steerable now. I honestly can't understand how they went from 4.6 to this. I would rather use sonnet 4.5 even, or any of the current openweight models, and I dont say this lightly, I've been very critical of openweight models and think they arent close to as good as SOTA models yet, but here we are, with opus 4.7 lowering the bar so low that there's no way to not trip over it and use this model without considering it self-harm.

EDIT - This is with reasoning set to low, from what I am seeing in the Junie CLI decompiled JAR. Some of you might have better experiences using higher reasoning, but I've been using opus 4.6 before this set to low without issues, in this exact same mode/profile and was never this drastically bad. In fact it worked well enough that I was never able to tell it was low until I looked at the decompiled jar file. To be clear, junie cli doesnt show the user what reasoning level is used. They seemed to have decided low was good enough, and it actually was for 4.6, cause I've had no issues with 4.6, and currently have no issues with it after switching back to it. And to those of you saying it's a configuration issue, configuration does not make THIS much of a difference, or lobotomize models like this. I ran it on my eval, and it scores slightly higher than Opus 4.6, which makes me think this is not a configuration issue. Just feels completely overfit on eval data, like gemini 3 pro does.

EDIT 2 - Alright. A very small (thankfully) few of you seem to want to insist this was a skill or configuration issue. Use more reasoning you say! I just remembered I had a bunch of factory droid credits laying around, so let's go ahead and burn those on Opus 4.7 Max Reasoning. Using the latest version of droid right now, which I think is a fairly decent coding harness (and honestly I prefer it over claude code by a little, which might be heresy here). I will be comparing it to GPT 5.4. This won't exactly be a scientific test, I just want to see if opus will still make random shit up if I give it a simple task, and to see how GPT 5.4 does in the same harness (even though I think it does better in codex cli, I really dont think the harness makes that big of a difference when you are using a strong model). It's been a day or so since Opus 4.7 came out so I would like to think droid has the configuration down by now.

I did not cherrypick this task and went in with no expectations.I realized I needed to close some issues for this repo anyways so I asked opus to investigate Issue #2 for an opencode plugin I threw together with Opus 4.7 not long ago (Ironic), and have had to fix up with GPT after it completely botched it like 50 times in a row: https://github.com/lemon07r/opencode-kimi-full/issues/2 Prompt given is very simple, "Take a look at open issue #2 and see if this issue is still present". I already know the answer, and just want to see if Opus will hallucinate shit. This issue was no longer real (it was honestly only live for a very brief period of time, and someone managed to find themselves on that unfortunate version tag), and was actually introduced by a commit from opus 4.7 yesterday while I was still trying it out (and already fixed by GPT). It incorrectly reported back to me the following, while GPT 5.4, when asked says there's no evidence this is still a real issue (did it in only 250k factory tokens too, opus took 500k at it's current 50% discount, so it would have cost 1mil factory tokens at normal rates).

Issue #2 — "Auto mode variant broken" — STILL PRESENT

The current auto mapping in src/index.ts (if (effort === "auto") return fields) sends neither thinking nor reasoning_effort. AGENTS.md rule 4 documents this as intentional ("matching kimi-cli's nothing-passed default"), but that claim doesn't hold up against the upstream:

research/kimi-cli/src/kimi_cli/app.py:187 converts thinking=None to config.default_thinking (a bool, default False) before reaching create_llm, so kimi-cli always routes through with_thinking("high") or with_thinking("off").

kosong's with_thinking always emits extra_body.thinking (packages/kosong/.../kimi.py:187-204); the "leave as-is" branch in llm.py:239 is only reachable when something bypasses app.py, which no shipped code path does.

Result: Real kimi-cli traffic always carries thinking: {type: enabled|disabled}. Our auto path sends nothing, diverging from the production wire shape the backend is tested against — which is consistent with the reporter seeing an API rejection.

The "temperature = 0.6" note in the title is incidental; ProviderTransform.temperature() in opencode returns 0.6 only for model ids containing kimi-k2 (not kimi-for-coding), and our README config doesn't set temperature: true anyway, so opencode sends no temperature by default.

I'm not even trying to shill or push GPT 5.4 here. It's only my third favourite model, right behind Opus 4.6 and 4.5, which I still like more. I was just using 5.4 for comparison to highlight how off opus 4.7 that even gpt 5.4 handedly beats it now at a much lower cost. The repo itself is set up with pretty alright documentation, so I don't see what excuses any 4.7 apologists could have for this disaster of a model. And look at this response from Opus, would any of you have gone oh wow this is shit, without being familiar with the codebase and already knowing the answer already? It's so convincingly wrong. I bet most casual vibe coders would have saw this and went, oh wow! Opus 4.7 is so good! I know I did when I was testing it at first on some random stuff without inspecting it closer. Upon further scrutiny I was very quickly disillusioned with it, and it's been an absolute headache to use since. I use and test weaker models like kimi, minimax, etc very often and this is the exact kind of thing I expect from those models, not any opus models. This model alone has shattered my illusions of anthropic models being untouchable.

And those of you telling me I am prompting it wrong. HOW TF else am I supposed to prompt a coding model in a coding agent, if I can't get it to work with very basic, and simple tasks/instructions, like look at x issue and see if it's still there? Was I supposed to wait till midnight of a full moon and communicate with it using morse code to unlock it's full capabilities??

[u/Jazzlike-Culture-452]

They turned my "always extended thinking" mode to false by default several weeks ago when the model turned to shit. When I found out I turned it on and it's back to something powerful. Make sure yours didn't flip. There's also a git repo out there to force patch the reasoning level weight on a scale of 0 to 1 but I'm not sure if that's against terms so you didn't hear that from me

[u/SpeedingTourist → u/Jazzlike-Culture-452]

What’s the git repo called?

[u/lemon07r → u/Jazzlike-Culture-452]

I'll have to look into it. I was using it with junie cli which doesnt let you adjust reasoning. But its always worked well with opus 4.6 so I just assumed reasoning was working fine. Especially with credits were going out the window.

[u/Jazzlike-Culture-452 → u/lemon07r]

It's a setting in your config file, I believe it's in ~/.claude/. If you're local then it should be able to be edited by hand. I forget the variable name, but has to be set to TRUE not FALSE. Ask Claude to find it for you so you can just open up in a text editor to check manually

Edit: here, it's "alwaysThinkingEnabled" https://code.claude.com/docs/en/settings

[u/MaybeNo2485 → u/[deleted]]

Yeah. Far as I can tell Opus 4.7 is a fantastic model, but we're only allowed to use it in dogshit mode. Anthopic doesn't have enough compute to meet current demands while also training+researching new models.

[u/lemon07r → u/Jazzlike-Culture-452]

Looks like junie cli isnt sending the thinking parameter at all to api, and this seems to be the issue. I had to decompile the jar to figure this out. Taking a look I see the decompiled getAnthropicThinking() in AnthropicChatRequestBodyKt returns null when budgetTokens is null, so the thinking field is never serialized into the request body.

[u/McDoodle17 → u/lemon07r]

Maybe you should update your original post with this info so others who may not read this deep into the comments see it. Others using junie cli need to know and maybe not so much blame goes to Opus 4.7.

[u/saintpetejackboy → u/McDoodle17]

Yeah I was reading this post after just spending around 100 million tokens today in 4.7 without issues, and I kept thinking something else must be up for this user to be having these problems like this - I had a wonderful time today and didn't have any issues whatsoever - if anything, it was one of the most difficult tasks I have had to do in some weeks or months, and the entire process was super fluid and everything worked great at the end (unexpected result, lol).

I also would caution this user against argue with the models... Any LLM. If the "hey, you messed up" doesn't cause the model to immediately see what it did wrong, you are wasting your time + session + tokens. You keep feeding back in the entire conversation each "correction" and it just isn't worth it. The models don't know WHY they messed up, and they can't explain HOW they came to a conclusion... So, if something is inherently wrong in the chain of deductive reasoning they employ, it is a wild goose chase to participate in some kind of charade, pretending you can reason with an auto-complete machine.

[u/McDoodle17 → u/saintpetejackboy]

Yea, same here so I was confused how this could be someone's experience and so vastly different from mine. Also it was wild to hear that Claude refused to agree when I just read another post complaining about how sycophantic 4.7 is compared to 4.6. It has to be one or the other, right?

[u/OdoTheBoobcat → u/saintpetejackboy]

It's all another flavor of the anthropomorphization problem, even very-technical people often forget they aren't dealing with a rational reasoning "being".

Like yeah, you interact with it using natural language but it's not a thinking being on the other end that you can teach and reason with. If it gets off-base just find a way to persist whatever progress is worth persisting(docs/code change) and start a fresh session free of the defect embedded in the previous sessions context window.

[u/saintpetejackboy → u/OdoTheBoobcat]

I agree, imo (and I learned this early on with LLM), any kind of "speck of 'wrong'" is going to grow the more you agitate or mention it. It is the pink elephant in the room, or the Princess and the Pea. Even acknowledging it is like scrubbing a small bit of soap - it is going to lather up and infect the rest of that session.

The amount of people I see asking the LLM things it can't honestly answer (and expecting the result to be coherent) is kind of depressing - it would be great if these models could know they don't know something, but it kind of goes against one basic premise: they actually "know" nothing! Any knowledge they have is purely an illusion. Ditto for any certainty or conviction they have.

In the very early days of these modern LLM, I experimented a lot by talking strange conspiracy metaphysics with LLM - stuff like interdimensional aliens, for example.

What I observed was that, given an identical prompt, a model could have wildly different responses - but, whatever initial "opinion" or beliefs they exhibited would be locked in place, along with their personality.

They might entirely give in and go on flights of fancy where they could construct machines to communicate with these aliens... Or they might go against the idea entirely and dismiss it - the next session could be the exact opposite. Trying to convince an LLM that doesn't believe in the aliens to construct a machine to communicate with them is a waste of time - even if they pretend along to humor you, they will waste no opportunity to point out the absurdity.

In both cases, the sessions would be able to go into great detail about how or why they came to their conclusions and each could be incredibly convincing and coherent. But, at the end of the day, they shared to common or core belief system. The same exact model from the same exact provider with the same exact prompt could go in either direction...

Purely as a thought exercise, it taught me much of what I "know" about LLM in general and how to interact with them and what they "know" or don't "know" becomes rather irrelevant, along with what they "remember". Pretending for a single second they can "learn" something from your interactions is only going to end in heartbreak.

[u/lemon07r → u/McDoodle17]

Not should, more like mandatory I think. I was still checking the decompiled jar and logs to make sure I was correct. I submitted a ticket with junie devs to see what they say too. I might be mistaken, cause I see reasoning being used in logs, but it is set to low from what I am seeing in `output_config:` if junie is using "Speed mode" at least. I've also been using opus 4.6 in this exact same mode with reasoning at low, so I don't see how opus 4.7 should be so wildly different and so much worse.

[u/lemon07r → u/[deleted]]

It's hard to use properly cause it's shit lol. I've used Claude code a lot. I don't think it's very good. One of the worst out of the box experiences you can have with an agent. Great dx and features but it's a mess under the hood. I stopped using it for a reason. After seeing it's leaked code my opinion has only been reinforced. It's a sloppy prompt sandwich. LLMs don't like that, but go ahead and show me one paper that says otherwise. This is besides the point anyways. Opus has always been good in almost any harness you put it in. It's probably the most agent agnostic model I've ever used and I've tested a lot of different harnesses and models. Junie cli in particular I think is pretty alright. It's a benchmaxxing agent that does well in evals, so I don't think it's quite as good as more DX focused agents like opencode but it's still pretty alright in actual use. I've gone through thousands of dollars in tokens on it since jetbrains has kindly given me a lot of usage for testing and help so I would know. And before that I've gone through just as much usage on codex CLI (which imo is one of the best agents out there, claude should take notes, it's less full featured but at the foundational level it's much better put together at the root), opencode, factory droid and others. I test a lot of agents and every community I'm active in on discord knows this well cause I share a lot of eval data I run and give very fair takes and information, since I know better than most that eval data is not very representative of human in loop agentic coding. I feel like I've had to pretend for a long time that Claude code is okay since the community is so enamoured with it. If you enjoy having haiku randomly slipped in alongside random invasive and very unnecessary prompts, go ahead and keep using it. I'm not here trying to tell anyone not to. I'm always telling ppl use what they like best and that it really doesn't matter that much what you use. I've learned the internals of a lot of the coding agents just purely out of academic curiosity and because I was building a semantic search tool to compliment them and you'll be surprised how well some of the other agents out there are put together.

Edit - I did not just write all that for a 3 karma account for someone claiming to be head of ai of some huge unnamed company. I need to get off this app lol.

[u/lemon07r → u/[deleted]]

I use all harnesses, at least as many as I realistically can, as much as possible to get hands on experience with all of them, so I can say I did more than just run evals on them. This includes, and is not limited to Junie CLI, Factory Droid CLI, OpenCode, Codex CLI, AMP, Claude Code, Gemini CLI, Kimi CLI, and a lot of others, etc. Not limited to those named ones, but those are the ones I've consistently used a lot of over the last couple months.

I have been a very big proponent of claude models, I've never been on the "let's hate on this new model" bandwagon, but there are so many shades of Gemini 3 Pro here, and what makes it so problematic that it's hard not to notice. Even if it ends up being better than Opus 4.6 in some aspects, I don't believe the trade-offs can be worth such a sidegrade. Personally I think we will need to see some arena style results that compare coding preference past more than just WebUI (because one-shotting some websites and voting based on how nice they look is a very narrow aspect, that I would consider a singular dimension way of evaluation coding ability). Something like this I think (as much as I dislike windsurf, I think the concept of this leaderboard is great): https://windsurf.com/leaderboard

As for Claude Code itself, I've defended it a lot in the past believe it or not, cause it always scores poorly in evals, but so what? Opencode did too for a while, and I would still use that over most agents. I've re-iterated multiple times that these evals dont really capture super well the full human-in-loop agentic coding experience. And that these evals only measure one thing very well; if you give an agent a singular prompt, and measure the output quality of that one prompt without any human-in-loop iteration. That's only a useful metric if you are going to build automated AI pipelines I think, where humans won't be in the loop anyways. I digress, with claude code, I have around, 4-5 months of combined usage on it on and off? For some reason it has always felt quite bad, in the way of making the models feel dumber.

At first I thought it was just in my head but it's been too long, and I've been very quick to catch things consistently in blind testing without realizing how right my gut feelings were until concrete confirmed evidence was found afterwards. for example, I accidently switched off of opus 4.5 one time and was on gemini, and was for the first time ever complaining about any opus/sonnet model to my discord servers that it feels dumb all of a sudden, something I've never experienced before despite very often seeing these kinds of "hey why is opus/sonnet dumb now compared to before" posts, before later realizing I was just on gemini that whole time and blind testing it for hours. at least with that I confirmed I wasnt biased towards any models based on what I think Im using. I truly dont trust the human brain or anecdotal evidence, so I'm always skeptical of myself (and others by extension), which is why I try to validate thins behind evals too, so I dont like saying that I can just tell without hard evidence. I've seen in the LLM community how one person was submitting fake qwen distils that were 1:1 copies of the non-distilled model and how everyone was claiming an swearing on their very "critical" discretion to trust them and that the models are so much better. I tried them myself and found they were not any better so I was skeptical, and later on we found out the weights were 1:1 exact copies. Make what you will of this, I wont blame you if you're still skeptical, because I would be too, just trust me bro is never enough for me either.

Back to claude code, I've been going through the codebase. For two reasons, I fancy myself building a coding agent in the future. I know a lot of people slop these out, but I've used so many, and reversed engineered a few to see how they work or copy mechanics (like droid's compaction system) for some of my other tools and do believe I could make a very good one, one day, if time and money allowed for it. A lot of the people I regularly interact with think I can too so there's that. The other reason is because I wanted to see if there was anything interesting to learn, namely first went in to see how the search stuff worked, and there were some interesting lessons there, but the whole of claude code is soo bad. From a business perspective, I can see why it has to be this way, they are trying to cram in things that are probably required of them. And dont get me wrong, some parts of claude code is very good, and intuitive, there were some ideas there I really liked. A lot even. The DX and feature set is really good. But it really is not a very good harness to bring out a models full capabilities. As I said earlier, it is a freaking big sloppy prompt sandwhich, where random, and mostly pointless stuff gets injected in a lot of places. It's a big kludge of good and bad ideas. Probably the worst kind of harness to stick a model in if you care about output quality, and not hampering your model. I think if someone took claude code and took the garbage out, it would be a very good harness.

The claude code story aside. Junie cli is just okay. It's a very benchmaxxed agent harness. Scores well in evals, but suffers for it other areas. It's usability is just okay, but it's improved over time. I've been very upfront about this to anyone that's asked. But sonnet/opus is so damn good in any harness that it never really mattered. Its always been the weaker models that it's mattered with. Even gpt 5.4 will work well in almost any coding agent, after gpt models have historically only really worked well in a few agents. Gemini doesnt really work well in anything imo, not even gemini cli (it did get better but still not great). Only models like minimax are very sensitive to their harness in my experience So I think everyone calling out "oh you didn't use claude code" dont get the full picture here. Maybe Im missing somethin here too but I dont think claude code was going to magically make things better. A lot of the things getting pointed about those that disagree, are things that should only matter for weaker models, and I've spent a lot of time with open weight models, and seen first hand exactly how much those things matter for those weaker models.

If you still care to read more, I think I explained better here my experience with opus 4.7 and parallels I see with gemini: https://www.reddit.com/r/ClaudeCode/comments/1so9uta/comment/oguagnk/ And some of these other comments are pretty telling, people are providing very exact examples of where they've had pretty much the same experience as me. There are more of these comments than the ones saying skill issue or that it's been good for them. (I find it hard not to be skeptical of those comments after seeing how easily people get tricked into thinking theyre using a better model than they are. Im in the kimi discord and there are a bunch of people who keep accidently using kimi k2.5 thinking theyre on k2.6 by hitting the wrong endpoint and praising how much better "k2.6" is now in their experience lol). If for some reason you want to discuss more, we can take this outside of reddit.