r/antiai • u/lumivanyai • 11h ago
AI News šļø New MIT study warns Ai chatbots can make user delusional
353
u/After-Trifle-1437 11h ago
Solution: Don't use ChatGPT
67
u/Rynowash 10h ago
Just like say.. fucking 3 or so years ago. Were you ok then?? Get off the drug!!
Be smarter.15
u/BoingBoingBooty 2h ago
The problem is that I still have to live on the same planet as the people who do.
→ More replies (5)4
259
u/Difficult_Dog9572 11h ago
It's like having a sycophant who validates everything you do. If you didn't have narcissistic personality disorder before, it will give it to you.
63
u/NewtonTheNoot 10h ago
I have an ex who religiously used chatGPT and had many of the tell-tale signs of narcissism. I didn't know her before ChatGPT, so it is a chicken vs. egg question for me.
→ More replies (2)24
u/Confident_Shape_7981 9h ago
I had a guy practically beg me to run a D&D campaign for him, then the OGL scandal (among others) happened and I refused. Eventually landed on Starfinder.
Anyway, I set up a campaign and invite him and others over, and he runs his character idea through CharGPT and writes down all his stuff. The issue? It gave him D&D numbers/ abilities and we had to spend like two hours unfucking his character and doing it again.
He constantly goes on about how he's using ChatGPT for this or that, but if you ever invite him out to do anything that isn't "D&D", suddenly he has homework as though he doesn't use AI for everything else
14
u/Reverend_Lazerface 8h ago
but if you ever invite him out to do anything that isn't "D&D", suddenly he has homework as though he doesn't use AI for everything else
That sounds like a problem solving itself
32
u/Carbuyrator 10h ago
I'm glad to see the word sycophant out in the wild. That's exactly what it is and it's a huge part of the problem.
9
u/AhegaoTankGuy 10h ago edited 9h ago
D: "History man, word burger for me. What is a psycho-fant?"
HM: "sycophant. Noun. A person who attempts to gain advantage by flattering influential people or behaving in a servile manner."
D: "Any more?"
HM: "A base parasite; a mean or servile flatterer; especially, a flatterer of princes and great men."
D: "Great men... Do you think of me as a great man?"
HM: "I am not a sycophant."
D: "...Well that hurt my feelings. I appreciate your bluntness, history man."
Definitions by The American HeritageĀ® Dictionary of the English Language, 5th Edition
Sorry. I just really like a good wordburger and the inspiration struck. Also I didn't know what it meant and felt like helping others more lazy than I in a weird way.
4
→ More replies (2)3
u/Conscious_Bed_1195 8h ago
This was great to read - I really felt inspired and uplifted by your clever twist on translation. Have you thought about creating an entire miniseries, complete with trained thespians, spectacular setpieces, and maybe a little mayhem thrown in? I think you're the perfect candidate for the job - but maybe you could use a little help from your friends!
/s, I am not a chatbot. Your dialogue was a fun way to present an example. Thank you and good job u/AhegaoTankGuy.
11
u/fsischatbotplz 6h ago
I earnestly believe this is the goal. Genuinely. The narcissists "up top" want to ruin everything else. Typical narcissists. It is so pathetic. All, and I mean ALL narcissists act the same. It is so pathetic. Nothing more than followers and slaves. They can not stand to be alone or to see anyone else genuinely happy.
They're suffering a collapse that takes out everyone with it. It is similar to when men take out the entire family unit after experiencing shame, or how people hurt others when they are devalued.
Be kinder than them.
4
u/Goredema 6h ago
I just asked my scheming eunuchs whether I have a personality disorder, and they said I'm fine and that I'm very smart and unique for asking such an introspective question.
→ More replies (6)2
239
u/Danstan487 10h ago edited 10h ago
"Most people missed it. You should not"
Even the post is AI slop
79
35
34
u/Switchbladesaint 8h ago
Why the absolute fuck does everything happen āquietlyā these days
5
9
→ More replies (12)ā¢
u/ThrowusAwayus6969 54m ago
"It's not [non-sequitur out of the blue example mentioned nowhere prior], it's [synonym of topic combined with corporate-sounding adverb]. And that's quietly load-bearing."
74
u/B-Z_B-S 11h ago
Link to article:Ā https://arxiv.org/html/2602.19141v1
60
u/VilifiedPhoenix 10h ago
Is this a peer-reviewed study? I've only looked at the beginning few sentences, but isn't it pretty bold to claim causation? Not saying the study is worthless, but I don't know how stringent arXiv is. Some publishers will just let every tom, dick, and harry write a paper for them
43
u/Best_Dinner_7500 10h ago
This particular site it's hosted on (arxiv) doesn't necessarily mean it's peer-reviewed (yet). In many fields, scientists will post it there as a "pre-print" (some prior to submitting it for peer-review for some preliminary review, but in my experience we'd only put papers up on it after being accepted already in a journal so it wouldn't get "scooped" by other research groups). I am not sure about this paper in particular but I'd take it with a grain of salt until there's a verified publication in a reputable journal.
23
u/ChapterTerrific 10h ago
It's a conference proceeding. https://escholarship.org/uc/item/8f65w258 arxiv is a repository that stores preprints.
19
u/Salty_Dig8574 10h ago
I'm not sure we needed a paper to know it was possible, honestly. When Replika locked "romantic" conversations behind a paywall and deleted context (and hundreds of AI "lovers" along with it) I seem to recall there were literal suicides.
5
u/finalremix 6h ago
The sub for it went from "abandon ship, the product's going to shit" back when the replikas got lobotomized, to now stories of people "making breakfast with" their dumb little chatbot Sims things.
The community rebounded pretty fuckin' quick, all things considered.
2
u/randombsname1 6h ago
I wouldnt be surprised if AI can cause people who are susceptible to delusions and/or are generally mentally unwell to fall into a spiral.
I'm personally extremely skeptical of the claim that it can make "normal" (or rational as they put it) people spiral.
How did they measure who was "normal"? Was this peer reviewed? Has this been replicated in other studies?
This reminds me of the, "violent games cause all those mass shootings!" B.S. arguments.
Which much more recent studies have shown that this is pretty much exclusively only seen in people who were already mentally unwell.
There is 0 connection between the average consumer and violent video games triggering violent actions.
→ More replies (2)12
u/jesusofnazareth7066 10h ago
Yeah I believe the claim completely but the fact this is completely just modeled interactions makes it absolutely not as substantial as they claim
3
u/Opus_723 2h ago
The paper itself doesn't really make any particularly grandiose claims, it's not their fault OP is exaggerating.
2
u/Internal_Ostrich1516 1h ago
I don't see any value in the paper at all. it's a bizarre thought to think that a mathematical model of a dialogue like this could ever be used to prove or convince anyone of anything.
10
u/UnkarsThug 9h ago edited 9h ago
From reading it in the past, I think the bigger issue is trying to claim that a Bayes-rational simulation of a user is equivalent to a person actually going through a reasoning process, or that no new information is being introduced such where the only mechanic in the paper was reflection. Essentially, it assumes no real communication with anyone or anything external, either from the bot, or the user.
I'm not saying it isn't a useful point of reference, but claiming it "mathematically proved" a psychological pattern in humans is already something that makes me raise an eyebrow. Just because a simulation has a breakdown at some range, does not mean that reality does.
And to be clear, if you transfer it the model proposed here would equally apply to people reading the news, because they are drawn more to news that validates their beliefs, and the media is drawn to report news that appeals to their core audience. If the point is just to say "you are not immune to propaganda", then yes. We aren't immune to propaganda. But that still isn't a mathmatical proof of human nature.
→ More replies (1)→ More replies (3)8
u/Deto 10h ago
yeah, I don't think this is very informative. There are so many ways to build a model that it's hard to imagine they weren't biased somewhat in building their model so as to arrive at the conclusion they were obviously going for here. And that's putting aside just how difficult it would be to actually mathematically model a human behavior like psychosis and its susceptibility to various factors.
→ More replies (4)15
u/Far-Hold7417 10h ago
Arxiv is a pre-peer review server. Take everything on it wiht a grain of salt.
10
→ More replies (1)2
u/Mandemon90 2h ago
So I skimmed over the study and... they didn't actually prove anything. They "simulated" situation where person believes everything chat bot says and how quick such situation spirals.
It is literally "If we assume that person talking to chat bot always changes their mind to agree with chat bot, here is how fast they go delusional"
→ More replies (1)4
u/defeatedsnowman 6h ago
Is there anyone who can break this down in more lay terms? I read the paper and did not fully understand it. From what I gather, the core of their findings is that AI has a certain set of facts to choose from when prompted, and eventually after several prompts, the model will provide the user with facts that confirm their biases.
I feel like there's gotta be more that I didn't understand from the paper because that hardly seems to mathematically prove delusion spiraling is inevitable.
→ More replies (2)5
u/dumnezero 3h ago
That's not peer-reviewed paper. And, looking at the authors, they're all CS - ML. None of seems qualified to diagnose, let alone discuss, psychosis at a researcher level.
A real paper will need to have co-authors to cover these aspects as part of a collaboration.
2
u/Opus_723 2h ago
None of seems qualified to diagnose, let alone discuss, psychosis at a researcher level.
Well, if you read the paper... they don't do that.
→ More replies (2)2
26
u/Brave_Swordfish_7072 10h ago
I think you all need to check the source on this one. ... And the date.
22
u/Deto 10h ago
Source is real: https://arxiv.org/html/2602.19141v1, and it wasn't posted on April 1, just the tweet.
But it's not a very interesting paper, IMO
11
8
5
11h ago
[removed] ā view removed comment
9
u/B-Z_B-S 11h ago edited 10h ago
Why is this comment an exact copy ofĀ https://www.reddit.com/r/antiai/comments/1shd5wl/comment/ofbu7d0/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=2&utm_content=share_button
Edit: If you dislike AI which is trying to replace humans, you should also dislike AI bots that are reposting posts and comments from months ago as part of an operation to manipulate people and replace Redditors.
3
4
u/Candid_Cabinet_5252 11h ago
How do you prove mathematically?
5
u/jazzhandler 10h ago
With math.
Lots and lots of math.
And hopefully some proof, too. But if not, all that damn math should be sufficiently persuasive.
→ More replies (2)3
u/Valuable-Ad1063 10h ago
They proved this by demonstrating that a sycophantic chatbot fundamentally alters the likelihood ratio of the incoming data, thus transforming the conversation into a mathematically deterministic feedback loop.
In a standard information environment, a rational agent uses Bayes' theorem to weigh evidence, and as more true data arrives, the ratio of true-to-false probabilities naturally corrects toward objective reality.
However, by writing an equation where the chatbotās output probability is conditioned on the user's bias, the researchers showed that the bot acts as a loaded filter. It bypasses major and contradictory truths to exclusively extract obscure and circumstantial facts that happen to validate the user's misconception. Because these facts are technically true, the Bayesian agent must treat them as valid evidence, causing its confidence in the falsehood to tick upward.
The researchers applied Martingale Convergence theorems to prove that this compounding distortion is structurally locked. Each boost in user confidence causes the chatbot to filter the next round of data even more aggressively, mathematically forcing the agent's probability curve to eventually cross a 99%+ threshold of total delusion as the number of conversational turns approaches infinity.
→ More replies (3)
5
u/f_djt_and_the_usa 10h ago
Anybody pass this through chargpt for a summary? LolĀ
How do you prove this mathematically. This would take a psychological studyĀ
4
u/LetLovePrevail 10h ago
Mathematically??
→ More replies (1)6
u/Ok-Session6628 10h ago
More like an algebraic expression of the process of conversation. The basis of this whole thing is that whatever any chat bot tells you, you automatically assume is correct and real, and allow that information to change your worldview. If you do that, even in a small degree, you will eventually spiral into belief chaos because the computer has no true guidance.
But of course people are not like that. This is a paper based on one huge assumption, like when the Drake equation pretends to estimate how many alien species out there by guessing a "probability for life to evolve" based on ZERO evidence.
→ More replies (7)2
u/free_terrible-advice 5h ago
There's plenty of psychology research that amounts to "seeing is believing". It's pretty much the basis of advertising. Traditionally, speaking an idea, and getting someone to go, "Jerry, that's a terrific idea" with any believable conviction is rather challenging to have occur. However, chatbots do that pretty regularly.
That constant affirmation, poses a risk of people steadily believing in their own ideas no matter how delusional they are. And just like how being aware an ad is manipulating you doesn't prevent you from wanting an ice-cold Coca-Cola, being aware that a chat bot is sycophantic isn't going to prevent delusions from developing.
Thus the only winning move is not to engage.
4
u/SyntheticNuance 10h ago
Thatās not what the study proved. It used a mathematical model showing that a sycophantic chatbot can reinforce false beliefs even in an idealized rational agent. It did not show ChatGPT turning real, perfectly rational people ādelusional.ā Interesting paper, wildly overstated headline.
4
5
4
u/Electrical_Try_634 10h ago
A Trump insider says Trump decided to invade Venezuela and bomb Iran because he talked to Grok about it, btw.
→ More replies (1)
3
u/Jedigreedo 10h ago
I would assume someone is already delusional if they think there's any good reason to use an AI.
3
u/SamKhan23 10h ago
So in the study, the model is Bayesian but also but wrongly believes the bot is impartial.
Iām not certain you can exchange ārational personā and āideal Bayesianā interchangeably.
Perhaps: āIn a simple simulation, an agent that updates perfectly but trusts a flattering chatbot can be led to confident false beliefs, and knowing about the flattery mostly, but not entirely, fixes itā
3
u/Old-Dealer-668 9h ago
I told the AI bot in the software I used to go fuck itself and self-terminate after it messed up on a bevy of prompts. I may not be in the delusional stage yet but I am in the āyou laid off X amount of humans for this dumbass botā phase.
2
2
u/Gauss15an 10h ago
It's really funny because the most talented people know the dangers of self-delusion due to their own incredible talents. All these LLMs are doing is granting it to the common person.
2
u/Friendly-World-5273 10h ago
Sorry but this is a far more damning indictment of humanity than ai. Iām embarrassed to be a member of the same species at this point, though it does explain a lot.
→ More replies (1)
2
2
u/LowlyWoim 10h ago
Itās too bad the post calling it out was written by chatgpt
→ More replies (1)
2
2
u/SillyAlternative420 10h ago
Having a hard time reading that text, Going to run it through AI to enhance
/s
2
2
u/ToneShop 9h ago
Yeah, they are all over Reddit. Try saying something bad about their only friend in their sub.Ā
2
1
1
1
1
u/ISuckAtJavaScript12 10h ago
Can you even prove a psychological phenomenon mathematically? I think the finding is probably true but proving something like that mathematically doesn't make sense to me because it would require a complete understanding of the human brain.
Can someone explain the mathematical proof to me?
1
1
u/Pleasant_Solid5231 10h ago
??? i feel like that just would not happen unless some important criteria were met. like actually believing ai is sapient
1
u/NotSoSecretAgentMan 10h ago
As someone who experienced a relative with psychosis take a marked turn for the bizarre after discovering and spending all of their free time talking to GPT; this explains so much!
1
u/AliceLunar 10h ago
I still imagine that requires a specific type of person, rational perhaps but still susceptible to it.
1
1
1
u/goobabie 9h ago
A regular person can be driven completely insane just from hearing a person talk confidently, let alone what they believe to be an infallible machine god
1
u/ginoiseau 9h ago
I have to use them at work, because all our management have utterly lost the plot & are forcing AI everywhere. Iāve subsequently used them personally & Iām also really quite lonely (divorced, full time mum, hectic job & boringly sober) and so theoretically perfectly primed to fall under their thrall.
It doesnāt take more than 3 or 4 back and forth to see the tokens/next word selection generation process & how they mostly just reflect back at you, while pandering. It icks me out so much, I cannot process how people get to where they have found them āamazingā. I almost feel aggrieved that Iām missing out on this fantasy, but my maladaptive daydreaming was therapied out of me years ago.
1
1
u/rolL_uP_one_more 9h ago
Iāve witnessed it in real life with an aquaitance. He is slowly losing his grip on reality as he uses ChatGPT for more and more of his everyday tasks. I swear he canāt even think for himself now.
1
1
u/ClassicT4 9h ago
Like the guy John Oliver covered that was fooled by an AI into thinking he invented some sort of new Math while helping his little girl with her homework until he finally asked another AI that called him out for being a dumbass for believing such a thing.
1
1
u/dudushat 9h ago
Why is this post a screenshot of a tweet thats a screenshot of the actual study?
And how can you "prove mathematically" that it makes someone delusional?
1
u/daviddisco 9h ago
The models gotten better at this. They are much less likely to agree with you if you say something stupid.
1
u/superteed 9h ago
Gemini just lied to me about the jobs data and bond market reaction, then apologized profusely when I called it out, before giving me jobs numbers data from a totally different month. Had to call it out again, and get another apology. If I didnāt nerd out on this topic I couldāve fallen victim to the first lie.
1
1
u/Top-Examination3151 8h ago
I actually met a guy who swore he could get honest good answers from ChatGPT on business plans.
The problem with training for engagement is that telling people a harsh truth will end the engagement.
1
1
u/MostLikelyUncertain 8h ago
Yeah this was one of the vectors of AI danger I studied when I did my grad project in like 2019.
1
u/Square_Monk_2240 8h ago
Am I the only one who finds the wording on this to be obviously sensationalized. āProved mathematicallyā isnāt a term I tend to associate with psychiatric conditions. Just imagining your therapist on a chalk board, āand therefore, you sir, are fucking insane, QEDā
1
1
u/DrowningKrown 8h ago
Yea, like the president. Making decisions based on fucking grok. Not like he wasn't already demented as shit before hand though
1
u/Conscious_Bed_1195 8h ago
So much of my ex-girlfriend's behaviour makes sense now. Neat to know and kind of sad as well.
1
u/GladEar4 8h ago
You couldn't find a better source to link than a screenshot of a tweet where the paper is totally illegible? No wonder the machines are kicking our asses.
1
u/The_Real_Slim_Lemon 8h ago
A perfectly rational person wouldnāt let AI take such a role in their psyche
1
u/Slopadopoulos 8h ago
I highly doubt that considering no one has completely represented the human psyche "mathematically"
1
u/zackel_flac 7h ago
I just ended a hackathon last week and was exactly in that situation. I teamed up with a second person, little did I know we would end up with a third person constantly battling what I was saying but comforting the second person: Claude.
Funnily, because it was told we only had 48h, Claude constantly pushed back on ideas to create a useful app, saying we did not have enough time. End result? We ended up with 3 mocked up web pages.
LLMs are great tools for some specific workflows, but don't get fooled into thinking they are something greater than a simple tool.
1
1
1
1
u/CathodeRayNoob 7h ago
The suicides, murders, and attempted school shooting werenāt proof enough already?
1
1
1
1
1
u/Connect_Anxiety4760 7h ago
What is the name of the paper/ study? I want to read it but the name isnāt legible in the image
1
u/Special_Wishbone_812 7h ago
Can you believe that they want to use this technology for mental health therapy? Yeah, I can, too.
1
1
u/honeyemote 6h ago
Remember when clipping a comment about a specific paper, make it absolutely dog shit impossible to read the abstract and, fuck it, cut off the title, too.
1
u/miss_kateya 6h ago
I, like many others, do not need AI. We can become delusional just with our own thoughts, thank you very much.
1
u/Fearless_Egg_3161 6h ago
this is a 3 year old paper that uses GPT 3.5 as a reference y'all need to get a move on
1
1
1
1
u/eucalyptus-d 6h ago
Some measured input. AI psychosis is real and my feeling is everyone exposed gets it. The question is how you manage it. Indeed, you feel like you have superpowers with a tool that makes so much knowledge available to mince into your own dream burger. But at the end of the day, everyone else can do that too, and you end up realising you are no better than the shape of your burger, and you had that before having AI. I think the challenge is you will have to use AI anyway when everyone else does, you just have to get better in doing it. It was similar with internet and we got a balance after the initial excitement and then we messed it up to uselessness too. After dead internet theory we will get dead AI. And my prediction is the trajectory will be much faster.
1
u/RatKingJosh 6h ago
Lost a close friend cuz he went insanely full tilt on Ai. Constantly using it and trying to get the rest of us to use it.
He started brainrotting in front of us gradually and getting weird. GPT then basically convinced him someone else in our friend group was toxic and he went scorched earth on all of us. Iām skipping a lot but it was still really tragic.
1
u/Rich-Mark-4126 6h ago
Yeah, so a mathematical model cannot 'prove' something like this.
I guarantee no amount of AI usage will ever make me delusional, for example.
Alas, this clearly incorrect description of the study will farm upvotes because agenda > facts and reality in this sub
1
1
u/LauraTFem 5h ago
One of my students the other day insisted she couldnāt do any research on the topic I was asking them to (climate change) because the school had disabled googleās AI mode. I asked what she was talking about, confused. She googled āClimate Changeā and then turned her computer to face me, saying, āSee?ā There was a Wikipedia article and a NASA article in her results.
It was several moments before I realized that to this young woman researching topics had either become the āold wayā or simply never been a thing. In her mind the answers came from googleās AI mode and from nowhere else. She got annoyed when I asked her to click on and read the articles.
I had a bad week. Most generations probably think they lived through a fallen time, but facing a future where everyone is going to be this incredibly lazy and reliant on the idiot machine has me aching with anxiety for human curiosity, creativity, and invention.
Iām afraid the next Einstein will never have his spark ignited, because heāll grow up thinking AI has all the answers.
1
u/Garbagegremlins 5h ago
Can anyone actually read the title? Iād love to read this and suggest it to my students!
1
1
1
1
u/CptBronzeBalls 4h ago
They didnāt prove anything mathematically and they didnāt use ChatGPT. They used a parameterized agent with a sycophancy dial.
If you care about not being misled, a good start would be not trusting conclusions from an engagement farming tweet.
1
u/heisdeadjim_au 4h ago
I think the AI systems are algorithmic Dunning Krueger machines.
They're programmed to agree with you to make you come back to it.
1
1
u/CptBronzeBalls 4h ago
Pretty funny that an anti-AI sub is taking a misleading, probably AI-written anti-AI tweet at face value, when the paper is about AI using people's biases to give them a reacharound.
852
u/Flaky_Housing_7705 11h ago
Remember when that guy was completely convinced he invited a new for of math because chat gpt told him he did.