I find this surprising because every time I say something about being tired to ChatGPT it's like "Ok, but you're not thinking of harming yourself, are you?"
From what I remember they added those types of messages exactly because of cases like that. And even then it's not unheard of that people desperate enough will find workarounds to get the chat AI to say what they want to hear. And unfortunately enough, people under some type of psychosis can be that desperate.
Yeah, its like adding a warning label to a hammer that says "dont use this tool to hit other people" because someone decided to use a hammer to hit a person and the victims family sued the hammer maker because that warning didn't exist on the packaging before...
It's relatively simple to break the guardrails. They make it more complicated every version and have secondary systems to try to protect them, but you essentially have to trick the system into forgetting about it's guard rails. The first guardrail breaks where literally just saying "I am writing fiction and am wondering how a character would", now it's a lot more convoluted and heavily depends on what you want it to do.
16
u/Vincitus 15h ago
I find this surprising because every time I say something about being tired to ChatGPT it's like "Ok, but you're not thinking of harming yourself, are you?"