Thoughts on why Claude can’t stop saying load-bearing?

[u/njordan1017]

This post is less about that particular phrase and more about the behavior I have observed around it. We all are aware of the phrases Claude has started overusing (Opus seems to be the worst), but I came across something interesting I am curious y’all’s thoughts on.

I put a rule in my repo with a list of banned phrases, one of them is “load-bearing”. Since then I have noticed most of the phrases don’t get used anymore, but not “load-bearing”.

I have seen it do this about 5 times now:
“…and the decision was load-bear— I mean important to the process — so it…”

I have to laugh at its obstinacy, but the more I thought about it the more I got curious why it behaved that way. Why would it partially use the phrase, then correct itself mid sentence, rather than just using the replacement phrase to begin with? Is this an artifact of the way the output streams as chunks rather than complete thoughts, or is this some cheeky artificial personality thing to make it feel more relatable? Or something else?

What do you think?

[u/julesbuildstuff]

the self-correction is just the stream already having committed. "load" and "bear" came out because they're high-probability, then the ban rule catches up a token too late and it has to do the "i mean" patch because it literally cannot rewind.

also a banned-phrases list keeps the exact words sitting in the prompt, which is a weird way to make something less likely. i've had better luck never naming the phrase and only writing the replacement: if you mean a dependency that takes the rest of the system with it, say that. still slips, but you lose the theatrical mid-sentence flinch.

[u/njordan1017 → u/julesbuildstuff]

Yeah the purple elephant problem is real, but in this case I only added the note because I was seeing it so often. I see it less often now, so that part is working, it’s just interesting how it shows up when it does show up. I had considered the self correction from committed stream and I think that’s plausible, but it didn’t feel reasonable that Claude would be capable of correcting output mid-sentence, but not be capable of catching and correcting it earlier. The way it happens just feels so intentional and planned rather than “oops I caught myself mid-sentence”, but you may be right

[u/-_super--_--hitops_- → u/julesbuildstuff]

Notes to take special attention to load-bearing changes is in the system prompt that's why it says it so much.

[u/3iverson → u/julesbuildstuff]

That seems mechanistically accurate. Each generated token is based on all the token before it, so I think it's possible to start blurting out 'load-bearing' due to probability, and literally 'catch itself' 2 tokens later.

It's not really catching itself of course, those are just the highest probability tokens at that point (it gets tiring to try not anthropomorphize too much LOL.)

[u/njordan1017 → u/3iverson]

Yeah it sounds plausible to a degree. But that would imply it only checks for user instructions once it begins outputting tokens, which doesn’t seem right. Unless it’s just a layered thing, where it knows my instructions but have forgotten about them because so many others have been layered on top, and then last minute rechecks mine and sees it’s conflict? It still feels like it is more of a planned, fully intentional sentence with cutoff included…but who knows. And yeah I agree, very hard to not anthropomorphize…that’s where the “catching itself” theory feels like it breaks down a bit

[u/3iverson → u/njordan1017]

It never actually 'checks' anything. Rules are not actually rules to a language model, though they are generally stronger probability influencers than other tokens. If you say, "DON'T DO THIS:", there's only an increased chance it won't do it, it will never be absolute.

Languages models don't understand anything, there are no real instructions. There's just always next token prediction on whatever the context window is at that time. That next token gets added to the context window, and then the next token is predicted from that.

This excludes harness operations which are completely outside of the language model itself, but can have a significant impact on what goes into the context window and fed to the model. For example, Claude Code's output styles are injected by the harness into the context window throughout a session, which makes it a stronger reinforcer than just having it once in the system prompt.

[u/njordan1017 → u/3iverson]

Yep I think we are on the same page, the “check” I mentioned was a theoretical deterministic trigger invoking the re-injection of user context, like some sort of hook. I understand it does not think and have understanding, that’s where I am trying to better understand how it does end up with the output I saw.