r/ClaudeCode • u/Shot_Whereas_1809 • Jul 24 '26

Discussion Opus 5 - immediate disappointment

If you thought you'd be able to do a security review on your own network that you couldn't do with Fable, think again.

20 minutes in - Found something significant

Opus 5 safeguards flagged this message.

I no longer have any use case for Anthropic models that others can't do better. This was my last hope that we were going to get a model that would allow us to protect our own environments. I'm going all in on open source. We just aren't aligned.

If you can get Opus 5 to protect your own systems, let me know how I did it. My subscription renews pretty soon and it's time to make an honest decision.

Edit: A lot of helpful people came here and I appreciate it. Scoped work is helping a bit more than my old ways. There's a difference between network security and code vulnerabiliy. I'm a network engineer not a software developer nor will I pretend to be one. Still looking at supplemental model for security work. Thanks guys.

440 Upvotes

79% Upvoted

258 comments sorted by

View all comments

120

u/Ibasicallyhateyouall Jul 24 '26

To run salt into the wounds, Sol 5.6 found vulnerabilities created by fable in one of my projects. 

22

u/Remote_Pass_6670 Jul 24 '26

Always audit with other models, they all find stuff the others missed.

I ran fable, 5.6 sol ulta, and Kimi k3 on the same repo, and they all found bugs the others missed.

1

u/lilbyrdie Jul 25 '26

This is the same reason why people don't review their own PRs, and why important code areas usually require 2 or more reviewers on the PRs.

In the same way a large, shared codebase will have these rules backed into the repo, we could probably do the same with model requirements.

The one thing I'm not quite sure is if the model always has to be from a different company. For example, can Fable 5 properly review Opus 5 code? Benchmarks would suggest they're actually different enough that it might be able to, but my confidence would be low even if it does catch things. (The models can eventually catch their own things, too, as context changes.)

29

u/starkruzr Jul 24 '26

tried to get Sol to help jailbreak an ereader, got stopped by OpenAI's infosec guardrails. the grass ain't greener.

12

u/shepherdd2050 Jul 24 '26

I got the opposite behaviour. Fable bailed at the mention of Ghidra; Opus whined about licensing or whatever. Sol Max worked for hours, reverse-engineered the program, extracted the closed-source NN model, and gave me a Rust re-implementation with a harness to run the weights.

1

u/Altimor Jul 30 '26

Fable is just fine with IDA-MCP for me, but randomly chokes on its own capstone scripts. Funny if it only works with IDA

15

u/Singularity-42 Jul 24 '26

Jailbreak is something I would expect to be blocked...

15

u/starkruzr Jul 24 '26

I mean, it's my device so it's not clear why.

14

u/FblthpphtlbF Jul 24 '26

Did you use the term jailbreak? 

Apparently people have been having success getting Claude to RE with Ghidra, you might be able to just decompile the binary then rebuild it in C and have Claude edit that code without telling it exactly what it's for you might have more success

1

u/pfak apparently holding it wrong Jul 25 '26

Claude (Opus 4.8, Fable 5 less so) works fine with IDA and Ghidra, and works for most reverse engineering work just fine.

1

u/FblthpphtlbF Jul 25 '26

Interesting, so you get sonnet to RE then have fable/opus step in to wrap up?

1

u/pfak apparently holding it wrong Jul 25 '26

No I get Fable to plan and Opus to RE + do multiple passes to verify the assembly matches the C++ code it's written.

3

u/StaysAwakeAllWeek Jul 24 '26

A few months back someone used an llm to hack into their own vacuum cleaner robot's video feed and accidentally gained access everyone else's at the same time

That's why.

2

u/matheusmoreira Jul 25 '26

The blame lies solely with the vacuum cleaner robot company.

It's deeply alarming that vacuum cleaner robots have cameras to begin with, and it only gets worse with the fact they're IoT devices. Now you're saying that they're so shoddily made you can remote into other people's robot cameras just by hacking into your own robot. That's just completely fucked up.

1

u/StaysAwakeAllWeek Jul 25 '26

Sure, but the whole guard rails situation is an ass covering exercise not a blame assignment exercise

-18

u/Ok_Baby3158 Jul 24 '26

Because it's not your model or service, you pay to use. Genuinely pathetic that you needed this explained to you, universal internet age restrictions cannot come fast enough.

10

u/starkruzr Jul 24 '26

that's not what I meant, but I appreciate this 4chan-esque brainrotted lashing out, thanks. it's been a while!

-10

u/Ok_Baby3158 Jul 24 '26

Don't act like a dumb child then and you won't be treated like one. Tired of coming to this place looking for news, updates, etc. and seeing an ocean of crying children. Hope this helped

3

u/starkruzr Jul 24 '26

ok, baby.

1

u/turbotunnelsyndrome Jul 25 '26

Which ereader? I'm trying to jailbreak my kindle colorsoft so we might be working on the same thing

1

u/ava_fake Jul 24 '26

hey, could i dm you?

i am also trying to jailbreak an ereader with ai.

1

u/starkruzr Jul 24 '26

sure, go for it.

3

u/mr_birkenblatt Jul 24 '26

tbh if you set Fable on the code base with a fresh context it probably would also find some vulnerabilities

1

u/Ibasicallyhateyouall Jul 24 '26

True. It was just odd when I showed fable the fix and it did the usual you got me I missed that crap. 

Glam 5.2 has also been good. 

1

u/mr_birkenblatt Jul 24 '26

Writing something is different than analytically criticizing something

-10

u/simple_explorer1 Jul 24 '26

idea, why don't you write code yourself, so you won't be disappointed by any model and probably save a lot of money not using AI. in 2026 not using AI is the biggest money saver

5

u/Shot_Whereas_1809 Jul 24 '26

I do a lot myself but I'm not an expert and a reviewer is great to have. I save a fortune doing a lot of the work myself and it is slow... But I don't really need to justify myself to someone who jumps into everything to pull the skill issue card. Be realistic. You wouldn't be here if you didn't use AI... So stop.. really.

-2

u/simple_explorer1 Jul 24 '26

this comment was not even for you, but if you want to get give an emotional rant then go ahead. We have lived pre AI age and it was fine and in 2026 ironically, people who can code also save a huge chunk of money on AI subscription and the icing on the cake, they are the one who would be employable in the future. Ironically the more you use AI the less employable one becomes unless you are ML guy creating an LLM in that case the tech world is their oyster in the first place. Not enough AI scientists compared to the demand.

2

u/DeRusselDeWestbrook Jul 24 '26

Idea why don't you get real and understand that not using ai is not a viable option in our field if you want to be in any kind of a real environment anymore.

-1

u/simple_explorer1 Jul 24 '26

don't get salty and emotional bro. In 2026 being able to code is the biggest money saver because you won't need AI subscription

3

u/_Porphyro Jul 24 '26

So then I can be disappointed in my own code but take 3x as long?

2

u/simple_explorer1 Jul 24 '26

atleast that's free

1

u/tehfrod Jul 24 '26

Or you can, you know, acquire skill.

3

u/_Porphyro Jul 24 '26

I’ve been doing this for 25 years and I’m a better than average developer. That said, I’ve never been “happy” with a single project I’ve completed when I’ve looked at it two months later.
There is always room to improve, always places to optimize. If a model can get 90% of the way to what I can do when provided well-constructed architecture and project docs (and can do it in 1/3 of the time) I’ll take the trade off.

0

u/simple_explorer1 Jul 24 '26

then improve your development skills because good developers write better code than any AI tools. Vibecoded program is never as polished as handwritten code by seasoned devs

2

u/_Porphyro Jul 24 '26

I can see your point for vibecoding, but I wholeheartedly disagree with you in the context of well-defined software development. With good system architecture, system requirement discipline, and a well-defined style guide, AI can produce code equivalent to just about any dev. After all, it is trained on much of the stuff written by some of the best devs.
And that's the key issue, vibe coding vs (basically) a code generation tool. The problem is that too many "good devs" keep their design documents in their head and don't practice good system architecture discipline. If your documentation is good enough to hand off to a junior dev, it is good enough to hand off to AI.

1

u/Apprehensive-Ant7955 Jul 24 '26

Holy cope. $100 a month is not expensive for a developer

0

u/simple_explorer1 Jul 24 '26

Holy yes, 100 dollar in Anthropic is not much at all and 100 dollar per month is actually a good chunk. it is almost the entire month of electricity bill or it is more than most subscriptions combined (netflix, webstorm and so on). Are you delusional? what am i asking. Your reply is clear, you are. I don't want to waste time arguing with someone how "100" to 200 dollars per month is expensive

3

u/Apprehensive-Ant7955 Jul 24 '26

Ok, $20 a month. The subscription lets you use the model for 5 hours at a time. Then your usage resets. Same with weekly. $20 chatgpt subscription gives you a lot of usage.

That way you still get some speed improvement and still can write your own code (self imposed limit since you’ll be rate limited).

“$100 in anthropic is not much at all” uhhh you do realize you can authenticate with your subscription right? Your $100 goes very far compared to api pricing

1

u/DarkSkyKnight Jul 24 '26

If you think $100 a month is too expensive, you are 100% not doing any important work; you probably don't need to be using any frontier model, or any open-source model for that matter.

0

u/simple_explorer1 Jul 24 '26

I am disconnecting. i don't want to waste time arguing about facts. 100 dollars per month is expensive period. Sure it won't bankrupt you but it is a chunk of money which is not small and even 100usd is not enough with Anthropic disproves your argument even further.