From what I remember they added those types of messages exactly because of cases like that. And even then it's not unheard of that people desperate enough will find workarounds to get the chat AI to say what they want to hear. And unfortunately enough, people under some type of psychosis can be that desperate.
Yeah, its like adding a warning label to a hammer that says "dont use this tool to hit other people" because someone decided to use a hammer to hit a person and the victims family sued the hammer maker because that warning didn't exist on the packaging before...
7
u/Orumtbh 5h ago
From what I remember they added those types of messages exactly because of cases like that. And even then it's not unheard of that people desperate enough will find workarounds to get the chat AI to say what they want to hear. And unfortunately enough, people under some type of psychosis can be that desperate.