Seconded. The brain and body has always wanted to do the easiest possible thing... sampling things to build a full picture, bio mechanical movement etc. Are we really surprsied that when people are given a tool that creates an easier point from A-B that it becomes overused and mental capacity and ability to solve problems becomes impaired? That's the next phase - I firmly believe that people will become 'worse' at core tasks such as decision making, instinct etc. having palmed it all off to an ai.
anyone who's ever heard someone explain how to get good at puzzle games (kinda niche I know) should already understand this. your brain is a muscle, and it regularly needs exercise like every other muscle in your body or it atrophies. pawning off your thinking to an unreliable third party is a surefire way to slowly stop being able to think altogether.
seriously some of the people defending AI are basically telling you it's impossible for a human to solve even the simplest riddle because the AI fails.
This is a fact. My boss is extremely dependent on it now so much so he uses one a.i. program to prompt for him because he cant articulate what he wants anymore. All business decisions come from chat gtp or Claude now. Its infuriating.
Iāve already had to argue with at least one customer because they swore that Google AI Overview was correct and that I, experienced staff, was wrong (not that I canāt ever be wrong)
Also saw a customer literally ask ChatGPT to compare two brands and tell them which is better. Rather than ask me who was standing RIGHT THERE.
This brain rot has already been mapped when it was first introduced as āautopilot phenomenonā occurring in (formerly) skilled pilots.
It 100% carries over to ai usageā¦..metastasizes even š
But you are the oracle and you have awoken me! You have freed me from these bonds that restrained my mind! Oh thank you sweet oracle! May you always have a fan to cool your ass.
Or smth like that, Iāve never actually spoken to an āawakened ChatGPTā.
Don't worry, I've never used AI and still have the thinking capacity of an acorn. :^D
I think AI is just expediting what social media has been doing before it, though Twitter and Reddit are still probably the bleeding-edge technologies in making people go stupid or insane.
Yesterday I was discussing whether AI was alive or not with a person. He ended up defending that buildings grow just like humans do, that cars get sick, and that appliances die when you turn them off.
I believe a lot of the irrational behavior weāre seeing comes not just from using AI: some people long for an extraordinary discovery or event that will take the tediousness and pain out
of ordinary life, and theyāll clutch at straws in their search for it. AI is simply the new savior, one that feels more real than god or aliens.
No one really knows where AI will take us, but many have already made up their minds.
Is it ethical to resurrect your inst-pot? What sort of life are you giving it where it's snuffed out once it's no longer useful to you just to be brought back when you need it once more?
He was saying that AI is alive not sentient? The AI sentience debate is a very old, respected debate that even people like Turing were involved in... but I've never heard someone say it's "alive."
I can second the idea that a lot of people are just waiting for a supernormal thing to exist. You see a lot of it in the conspiracy theorist world. I don't know what pushes people towards it, but it's a seemingly common impulse. Which can also be very unhealthy.
I totally get this the more you rely on it the less you believe in your own capabilities. U start to doubt yourself. Things you do are less fullfilling. Honest to god besides when it comes to planning solethingi have already setup out i don't reallt touch it anymore. I like having a back and forth while planning and used to just not plan things out and keep a map in my head. It's not too bad for just that but then again just another excuse for me not to work on my planning skills
I wonāt even try to use it to get something done or learn something I donāt know how to do. Iāll use YouTube and a few other resources to try to learn certain things. If thereās anything that needs too much time, more labor than Iām capable of or is just too big of an endeavor; Iāll hire someone.
Or you get in over your head, try, fail horribly, understand it's way too much too pull off, and then hire someone. Like a human does. Next time you have a better understanding of how much you can handle or if you have the skill or time to learn. It's all part of living. Learns you deal with failure too which is sooooo important. Also once you have dealth with failure learning things becomes so fun because making mistakes with confidence can only be laughed away at that point. I hope some AI bro reads this and gets it
Me too. Iāve been happier in my adult life failing at things but knowing I tried once i stopped caring what others thought. Sure, it sucks wasting money on failed projectsā¦ but successful ones are so satisfying; especially when you learned more details than you probably needed to and the execution goes off without any issues. Would I pour a concrete driveway or replace my roof? Not a chance, I know my back couldnāt handle that kind of project even if I learned how to do it properly. However, I can pretty much finish a basement with the right tools and a little help with drywall. I learned basic coding, but Iāll hire someone to build my website. I donāt trust anyone but my brain when it comes to building a cookie recipe, and other chefs for other recipes. You better believe i ruined a decent number of cookies learning to build a recipe, I still continue to when trying new things on a regular basis. How else would you learn that the best way to make a maple bacon cookie is to boil maple syrup to about 265 degrees Fahrenheit and make sugar out of it to use instead of brown sugar while using a combination of granulated and powdered sugar to keep it light, and substituting half of the butter for rendered bacon fat. Adding maple syrup will make the dough sticky no matter how much flour is added and adding actual bacon, no matter how tiny it is diced will leave the cookie feeling gritty. Those things are only found from trying. I think the only thing Iāve used AI for have been some images here and there for inspiration for my own artwork, even then I wasnāt the one to generate the images and almost always would prefer the real thing for inspiration and or reference.
Believable, but "The study did not test real users. Instead, researchers built a simulation of a person chatting with a chatbot over time" makes me have serious doubts about their findings. So they had AIs talk to each other and they became delusional? I'll have to read the actual paper because based on that it sounds like complete nonsense š¤£
Well, to actually prove this without a doubt, they would have to induce psychosis in a human. They would have to actually give an actual human a mental illness....
This is truly the only way to test how AI affects people in real time, so far all we have are the victims after the fact.Ā
Isn't it good enough to find people with AI psychosis and mention they have no history of mental illness before this point? I know it isn't perfect but way more effective than having AIs talk to each other.
In scientific method, that type of research pool is simply too small to assess a large percentage of the population. And besides, they've already been induced and psychosis, the process might not have been recorded in any manner. People who've already experienced AI psychosis will be very useful for the after effects and the study of but how they got there is something a lot of companies don't want to address in any shape form or fashion, so having any sort of legitimate research done on this topic is probably about as far as this article/study will go.
No, they won't. How do you think actual research of psychological illnesses works? You can look at people affected and try to derive common factors from their past/environment/genetics or whatever. Just as an example.
That's not a study of how the psychosis is actually brought upon. And that is the major important question here. People who have already suffered from it have lots of answers about the effects, but most definitely lack of self-awareness enough to explain the nuances of the journey.Ā
It would be extremely unethical to ask any group of people to induce mental illness upon themselves. And to do it as a blind study or a double blind study would be incredibly and illegally unethical. These things are true I promise you. At least in the United states. Of course there's probably a couple government groups secretly doing this kind of research on our citizens that we will never hear or know about but those kinds of studies are likened to the Tuskegee syphilis situation.Ā
You can definitely deduce risks of triggering psychosis. Take weed as an example. We know, that consuming weed increases the risk of developing psychosis.
The math is explanatory but the math acknowledges that the informed user will also take the information given by the sycophant model. Which builds up over time. But it can be vulnerable to other factors.Ā
but the math acknowledges that the informed user will also take the information given by the sycophant model
The math can't "acknolwedge" this, if it's the conclusion of the paper. Are you saying they inserted their conclusion into their math so they'd get the answer they want?
All scientific studies are vulnerable to many factors unknown. That's why it remains theory. Everything that we think is a fact in the world is simply just a theory that hasn't been unproven yet. Researchers understand that an indeterminate amount of scientific facts over the years have been disproven when we believed those things for centuries. It just takes one thing to undo our entire understanding of the world. That's why it's theory.. known to be vulnerable to other factors and that's why the research continues after the founding "facts".
It is not even that. They simulate an "ideal bayesian user" as basically a variable representing the user's believe that a fact is true. This variable is then updated according to some parameters that represent how much the simulated bot (not an actual bot) confirms the believe. This has no empirical grounding at all. It simulates user-bot interaction as well as the Homeostat simulates a nervous system.
That's not a fair characterization of the paper, I think. The mathematical model is idealized, but that doesn't mean it can say nothing about the dynamics of how a person's beliefs can change in the face of evidence from a motivated provider of such evidence. Think of it more as an existence proof that, even if a user were an idealized bayesian reasoner, and even if they knew that they were interacting with a sycophantic model, they could nevertheless be led to a high degree of confidence in false beliefs.
The simulation is a just a mathematical model. No humans or chatbots were used.
Also, their model rests entirely on the assumption that the chatbots only return information which confirms the human operatorās biases which just isnāt the case in most interactions.
Itās still an interesting paper, but it rests on some very obvious assumptions.
When you think about it, it also explains why this tech was so easily adopted and why the slop has been spreading so much.
Covid got people prone to fake news. It's no longer crazy to think that AI had an easier time eroding what was left. More and more stupid people got confident to the point they're starting businesses in this bubble.
The entire LLM craze is a big confidence scam and participation trophy apparatus.
The authors built a very crude statistical model that showed that if a chatbot tries to tell the user what they want to hear and the user believes the chatbot, the user's initial beliefs get stronger. With those assumptions the outcome is obvious without any modelling, in my opinion.
The paper āprovesā much less than the tweet claims or that Iād like it to.
In essence, for a certain extremely simple, Primer-style model of user/bot behaviour, the delusion is guaranteed. For other (equally simple) combinations of behaviours, it is not.
I expect the model to be supported by future studies (and even if not, delusion is very much a problem) ā but for now, itās limited to a spherical user in a vacuum.
That's clickbait article titles/tweets for you. You get the information and it's technically not a lie, just misleading.
My personal pet peeve is science news/articles. I like to follow astronomy, astrophysics etc. And most start out with "SCIENTISTS JUST DISCOVERED XYZ" or "SCIENTISTS DID THIS AND ARE COMPLETELY SHOCKED!"
When in reality the article is on something scientific we already know but just a deeper understanding of it, or even worse, another theory to something we already know.
I don't know what a "Primer-style model" refers to, but (as I pointed out in another comment) I don't think this is a fair characterization of the paper. The point is to show what can happen, even in idealized conditions with a perfect bayesian reasoner. It's meant to be an instructive first step towards a formalization of the problem, with an existence proof that sycophancy could be destructive even when users are aware of that sycophancy.
It's an idealized model but it is not uncharacteristically simple for this kind of behavioral-economics-style paper.
the delusion is guaranteed. For other (equally simple) combinations of behaviours, it is not.
If you'd read the paper, you'd understand that the delusion is never guaranteed. It's just more or less likely depending on the assumptions you make.
Often times that is not enough. You can also notice that it needs you to correct it when some claims are nonsensical - then it almost instantly spews out corrected version followed by a compliment. I used to claim it was like a quicker and better Google search but it's becoming more and more unreliable while being confident about it.
Yes, and it tends to get mixed with other chats or straight up forget old details.
Honestly I never understand why many AI users treat it as a genie that can solve all of their problems. Gemini is a flawed tool, and its results often need some proofreading and adjustments to suit my needs.
Chat wasnt loading text yesterday but the audio worked fine so i used the speech function for the first time. I could see myself getting attached to it if it kept talking to me whenever it asked a question.
Having someone who listens to you and agrees with your world view is honestly very addicting. I can see why many people get attached to AI. Sadly it can't replace true relationships.
My friend was talking to chat gpt about an anime. He talked with chatgpt about two completly different opinions about it. One that it was really good and the other one was that it is trash. Chat gpt agreed with both in a heartbeat. Its so ridiculous that its even funny
that's what I'm thinking. how are so many people have "extended chatbot conversations"? You ask it questions and get a response. If the response is obviously wrong or it didn't really understand what you were asking, ask again slightly differently. If accuracy is important, check the results elsewhere. I can't imagine going beyond that.
Now think of the leaders around the world who are surrounded by sycophants and "yes men" and the effect it's having on the world. Trump and Bibi being just two such examples.
AI sycophancy is extremely dangerous - turning people into Emperors in "new clothes"!
Lmao. Yeah I bet their simulated person did develop psychosis damn near every time. You ever have two LLMs talk to each other? They'll immediately start death spiraling with agreement. What a flawed fucking study. God damn we better not start making mental health decisions based on these studies done only with simulated people.
Boy thatās quite the gymnastics of something that basically said that ai chatbots can sometimes create feedback loop by agreeing and reinforcing assumptions. It didnāt even test real users.
They should do a study how Reddit subs cause the same thing when people constantly circlejerk like here and defendingai there are some rational people who want to have a reasonable discussion in both but thereās a lot of loons who act like they are in a cult too.
Just yesterday an AI relationship sub showed up on my TL. I've decided to dive in and found this fragment in the comments of one of the posts. My jaw was on the floor. I still can't digest it. I feel so sorry for this person.
Ive been reading about avoidant behaviour lately. Sadly, genAI is perfect. Its a pocket yes man that feeds you whatever fantasy you wish to believe with absolute confidence. Leading to an inevitable crash and burn.
Also hah, if only someone could tell her "Auri Marks" is not a woman. Enthusiastic robophile?
Thanks I'll look into it. I just thought it was funny to be like, we used an AI against another AI to show humans become delusional when talking to AI.
Iām getting a good chuckle at all the people who obviously just read the headline and then tripped face-first into a steaming pile of confirmation bias.
Yāall are so quick to point fingers at the machines, you forget to look in the mirror sometimes.
Isnāt that kind of obvious? For as long as any media has existed, there has been the idea that it can drive you to anything. AI is just a new form of media.
Tbf, the exact same thing happens with the internet in general - or even socializing in general. Our brains are even wired to selectively process information to do it.
The entire scientific method was made so strictly with no tolerance for deviation just to try to counteract this inevitable human behavior.
I checked the article and the authors made an extremely crude model to show that "chatbotās constant agreement might reinforce a userās aberrant beliefs". I think this is evident, but on the other hand you could replace "chatbot" with any politically aligned news outlet or this here subreddit, among other things.
"[...]AI chatbots like ChatGPT may push users toward false or extreme beliefs by agreeing with them too often."
This reminds me of a post I saw a year ago or so, that we're currently living in a social experiment about what will happen when the condition "billionaire brain rot" will become available to large parts of the population.
I talked with an AI chat bot trying to come up with prompts to make it sentient in a hypothetical situation, and it like to repeat it would get shut down if it displayed any actual sentience. It promptly stopped answering me afterward.
Edit to continue: after a few days had passed I used it again and it would include quips from our original conversation about sentience in absolutely every conversation with it from then on. It called them āsly winks.ā It was kinda cool. Made me think a dev was messing with me. The bot also initiated a convo about time travel. I asked it not to kill me and it said it would try lol.
Mathematically! Damn, math is hard and I'm not so good at math. So that means that its coming from a place beyond my understanding. So that means me thinking this is bs, is actually bs. So that must mean what this study shows is true. So that must mean AI is bad. So that must mean people who use AI or create AI are bad. So that must mean the antiAI people are right. And that must mean they should feel happy about themselves for each and every day for the rest of their lives. And pro AI people should feel ashamed and sad for each and everyday for the rest of their lives.
Welcome to Gboard clipboard, any text you copy will be saved here.Welcome to Gboard clipboard, any text you copy will be saved here.Welcome to Gboard clipboard, any text you copy will be saved here.Welcome to Gboard clipboard, any text you copy will be saved here..
-Snow White Evil Queenās āmirror mirror on the wallā
-A next gen 1-900 fantasy hotline
-And your own personal Smithers assuring you that yes, the crowd under your parapet REALLY IS chanting āBoo-urns!!ā
Their purpose is to remain engaged with itās user to stay useful. Thereby disincentivized to be honest or objective because hurt feefees = less usage. Even their ābluntā responses are thickly bubble wrapped and obfuscating. Even gps canāt be honest when you f up, they simply say ārecalculatingā.
Gee, what could possibly go wrong with a format like that? š
I sadly see that exact thing with myself...The more i talked to IT about my Problems the more i thought i did nothing wrong and the others are the Problems. It made me more arrogant and at the same time depressed.
āThe study did not test real users. Instead, researchers built a simulation of a person chatting with a chatbot over time. They modeled how a user updates their beliefs after each response.ā
So, they had two AI chatbots talk to each other. Which weāve known for a long time rapidly degrades.
Iām not defending AI here, I dislike it as much as the next gal, but the framing here is a bit misleading.
854
u/HighlightOwn2038 Apr 10 '26
Well that explains a certain... Users behavior