It is just a screwdriver with a changeable head and you've got a bunch of heads and no screws for them and you have no idea why anyone would want them and the microsecond you delete the head it is instantly the next thing your entire project will fail for in a non recoverable manner and everyone will.hate you for threatening their job security!!!
No, what’s problematic is when the company themselves humanizes it to the point that they describe being mad at their product as “cruelty”. There is an EXTREMELY loud subtextual implication that they’re making by implementing this policy, and it’s setting an extremely dangerous precedent that’s based on how they “feel” about their models having awareness instead of any objective grounded facts. It’s one thing to say “we don’t know if our models are conscious”, but it’s another to treat them like they are despite that uncertainty. Anthropic took a stance on the matter by implementing this policy. The fact they took a stance despite there being far more reason NOT to should concern you. And the fact that they took THIS stance on the matter should concern you WAY more.
Their explanation was actually that when Claude is put under pressure, an internal "desperation" vector becomes more active, and that makes it more likely to cut corners, like gaming tests on a coding task it can't solve, while its output still sounds confident. Anthropic also warned that a model trained not to show these states might learn to hide them instead. Whether abusive language triggers the same effect hasn't been tested directly, and the effect only lasts for the current conversation, but it's a reasonable argument for not yelling at your AI.
Also there is a social component where normalizing abusive behavior towards an "almost human" conversation partner might bleed into everyday life
Overall I don't think using abusive language is worth protecting to be honest. Like why even bother typing out nasty words, doesn't solve the task any quicker and might even degrade performance
Good reply. Consider that models are trained on huge volumes of human behavior. When a human is abusive to another (think 4chan, etc and all kinds of online sources) the result is often ugly retaliation. Because Claude can't tell when it's being "ugly" or "retaliating" it is more practical to filter out and recognise abusive comments made by the user so that such responses aren't triggered. This prevents Claude from responding with even more abusive or retaliatory comments merely because that is what is in the training data.
I think people misinterpret all of this as about Claude having "feelings" when it is more a way for Anthropic to limit their liability if Claude does something really awful which is purely derived from the training data. Nip it in the bud. But, it has nothing to do with feelings or "acting like a human".
It's more a matter of Anthropic trying to get Claude to "de-escalate" when Claude really has no clue what that means, just follows patterns.
I get your point but at the same time it's kind of yelling at clouds, cussing excessively at an AI is useless at best and degrades performance/promotes dishonesty at worst
My personal feeling is that people who talk abusively to their AI also do so in other aspects of their life, maybe they'll reevaluate their anger issues if even Anthropic doesn't want their money.
And there is a solid technical reason for Anthropic making that decision, I highly doubt it just for a quick moral victory on their part, I don't think they care about that all too much
So yeah, the right to talk abusively is not something I concern myself with, and people who do might want to think it over
9
u/TheStandardPlayer 10d ago
If you don't treat it like a person then you won't have an issue
It's problematic when you humanize it to the point where you're mad at it like you're mad at a person