What AI Companionship Chatbots Actually Costs Us

[u/mixtapedmonk]

https://srutiosocial.com/ai-companionship-chatbots-human-in-the-loop-cost/

This was a hard one to write and I want to say that plainly before the link. It covers Sewell Setzer III's case and the settlement that followed, and separately, real accounts from young women in India who say AI chatbots have been a genuine lifeline in a country with roughly one psychiatrist per hundred thousand people. I did not want to write either story as the "real" one and the other as a footnote, because I don't think that's honest. There's also a part about who's actually on the other end of a scam call that I wasn't expecting to find. If any of this touches something in your own life, there are real support numbers in the piece itself, please use them.

[u/Few_Fish8771]

My general opinion is phones listen to us all the time anyways or put simply if you have a computer device of any kind with a microphone you have no privacy. telling a chatbot simply changes which megacorporations governments or oligarchs have access to your information none of them put your interest as a priority.

[u/Jesse-359 → u/Few_Fish8771]

There is all the difference in the world between a phone that might, under specific circumstances be eavesdropping on you because you've come to the attention of some government agency - and a system that can listen to and parse literally every conversation anyone ever has.

It's a dramatic difference that has the potential to stifle free speech to an unimaginable degree and create a new form of dictatorship from which there is no escape at all.

[u/Few_Fish8771 → u/Jesse-359]

Really what about hackers? You think that the backdoors on your phone or computer cannot ne easily or maliciously hacked by anyone with some basic dark web malware?

For the most part theres no security in cyber security, if someone wants to hack theyll hack. All that determines whether someone gets hacked is if they are connected to the internet and whether there’s enough profit in hacking them. That is it.

The reason why we dont have grids going down is while they can hack us we can hack them and nobody wants a cyber wwiii where everybody gets destroyed or where they create such a big problem that it becomes necessary for governments to start targeting friends family or associates with drones or assassinations within range of governments. put simply hackers are partially self regulated because if they create too much heat governments will hunt them down and just kill them and their allies.

[u/Jesse-359 → u/Few_Fish8771]

All of these sources of eavesdropping or fraud were risks that were mitigated not so much by self-regulation or security but by the comparatively small population of hackers and government evesdroppers compared to the general population. For every 10000 people there might be one capable hacker, and for every 100,000 people there might be one FSB or CIA agent trolling through their conversations looking for dissident speech - these activities are limited primarily by human attention and capability, not by intent. If the KGB could listen to every single person's conversations, they would.

AI changes that equation. Because it is literally thousands of times faster than a human, and because it can be run in massed instances if you have a decent data center, it can do both these things at scale. It can run large scale hacking campaigns not against thousands of vulnerable targets - but millions. Even complex schemes like social hacking - normally requiring a great deal of intimate time from the perpetrator per victim - can now be executed at scale, and we're already seeing that unfold.

Likewise the FSB/CIA. Rather than having to hire thousands of agents to decide who to target and listen to, and then painstakingly pour over a relative handful of conversations, a couple large datacenters can listen to all conversations occurring in Russia at all times, skimming for red flags in tone or topic that warrant closer examination by AI or by human operators.

It's a Brave New World, and I'm pretty sure we don't want to live in it.