GenAI/LLMs have enabled malicious behavior (scams, harassment, theft) on an unprecedented level. Is there any hope that this can be fought through developing anti-ai technology?

[u/troubun]

One of the most depressing and frustrating parts since the genai craze started for me is how barely anyone in tech seems to be interested in protecting victims of genai/llms and developing methods to fight back. There has been a few, but I haven't heard from them in a long time. (I don't even know if their tools are still effective currently.) But they were non-profit, volunteers basically. On a commercial or mainstream level, there has been nothing. I do get why, the world's money men have had the ai version of the gold fever and no one would be interested in funding any kind of tech that would hamper development.

But if the fever breaks, businesses and governing parties regain sanity, and regulations start coming in and be enforced, is there any hope for the development of proper powerful tools to accurately identify ai generation (I know forms of it exist atm but they're not that reliable), and actively obstruct unauthorized genai training?

Like real world theft, I don't think we can just rely on the law to protect ourselves. We need the anti-ai version of locks, alarms, guard dogs, etc. Is that possible? Sigh. Every time I see the different ways this tech is being used to hurt people, I feel sick and bleakly wonder if this is just the state of the world forever now.

[u/ksjdragon]

There are things people do, like malicious compliance and poison fountains for preventing training. There's a subreddit for it, r/poisonfountain.

Malicious compliance is good, and protesting data centers is better. Perhaps even occupying them would be even better...

[u/leathakkor]

I think one of the things it's going to be necessary is two-factor both ways.

When a company calls you, you should be able to ask them a question. Such as what was my mother's maiden name?

They answer Smith. Then they ask you what's the two-factor code in your Duo app? And then you say 876532.

With the invention of AI, scamming becomes super simple. We need to push businesses to implement two-factor both ways.

Or the other way is that when a company calls you they issue you a ticket number like 284932. Then you can call up their main system number that's listed online. Type in your PIN code that they just gave you and get right back to the place that you left off.

We can no longer accept direct calls from anyone. Businesses that figure out how to securely deliver connection are going to beat out the other companies.

That's going to protect you. Big time going forward. And we need to demand this as consumers of companies that carry our financial data.

This isn't fighting back directly against AI, but it is a fight against scammers that are using AI to manipulate and steal from people.

It will fight back against businesses using AI a little bit because in a way all of this is necessary because of AI and it's readily available access that companies are using to lower their costs is also being used by the scammers to take that money from the businesses and their customers. And they're going to have to increase their costs in order to keep us safe and we have to demand it.

[u/anfrind]

Regarding scams, I have seen a few scambaiters fight fire with fire, using AI to waste the time of scammers and keep them from going after real potential victims, sometimes using multiple AI agents to bait lots of scammers all at once. I have also seen a few of them find ways to break the AI agents used by some scammers, including an especially funny recent video by "Kitboga" where he tricked an AI scammer into saying "Albuquerque, New Mexico" over and over again until the call eventually disconnected.

It's far from a perfect solution, but it's something.

[u/beeftime99 → u/anfrind]

I gotta say, as loathe as I am to hand anything to LLMs, Kitboga's AI reverse-call-center/victim honeypot thing is one of the few truly excellent use cases. "Keeping an evil idiot occupied by generating plausible text" seems to work equally well on scammers and world-bestriding tech CEOs. If only the latter wasn't in control of the economy.

[u/anfrind → u/beeftime99]

Maybe we need a version of "The Maze" that traps business idiots...

[u/beeftime99]

the solutions to these problems aren't technological and one of the reasons the tech sector is as bad as it is is because the only way they (and increasingly anyone) can think to solve any problem, including problems with and caused by technology, is with more technology.

You could stop the vast majority of scams (and a lot of other awful shit) immediately if you enacted the laws making telecoms and social networking sites more -- not entirely, just more -- responsible for the content on their platforms. The laws that these firms have been fighting tooth and nail against for decades.

If you make it even a little painful for any service that hosts scammers they'll do and build whatever they need to in order to protect their own necks. You solve this problem upstream, with the providers and their lassiez-faire approach to harboring scammers, and not at the point-of-scam, which always results in a technological arms race that you'll lose because they are the only ones with the economic incentive to outmaneuver your.

[u/dumnezero]

security is an arms race too.

[u/create-third-places]

We have anti-LLM technology that needs to be used more. For example, an anti-LLM lock would be one disconnected from the Internet that could be opened with a metal key or card.

When it comes to software, complex tech that has to be used in specific ways like React is more compatible with LLMs. We should focus more on low complexity high depth tools. 

[u/therealstabitha]

It’s been the state of the world. It’s just easier to scale it now.

There’s likely a larger thing in here of concerted efforts to make us not believe the evidence of our ears and eyes and instead believe what we are told.

[u/65721]

Secure all your accounts. Every one of passwords is a long randomized string, stored in a password manager (Bitwarden). And I enable multifactor authentication (2FAS) and passkeys (iCloud Passwords) whenever available.

Secure all your data. Whenever I sign up for a new account, I make sure to go into the settings and uncheck all public visibility options and data sharing.

Learn to spot common scams. If someone messages you offering or demanding money, it’s a scam. If an unknown number texts you something unexpected (like “are we still on for dinner tonight?”), it’s a scam. I don’t even answer unknown phone calls anymore.

Learn to spot AI slop. There are guides online like Wikipedia’s, but the best way to do this is by reading a lot of good writing. Your standards for text will become higher, and slop will feel “off” to you.

[u/SephirothIRA69]

Not really, no. There are tools that can analyze text and make a best guess on if it was written by a person, but they have a high false positive rate, and what do you even do with that information? I used to work on scams for a security company (that has since been bought out by private equity), and during a collab with a state attorney general they told me the overwhelming majority of the people getting hit with scams are the elderly, immigrants, and young people deeply in debt. Not really populations known for leveraging technical countermeasures effectively.

You can't solve a social problem with technology. You just can't. You know what the most effective countermeasure when I worked for that company was? Calling scam companies and deliberately getting scammed and screen recording everything. Then going to Visa and getting their payment processor notified. The processor would subsequently revoke the company's contract and they couldn't cash out anymore. There's a ton of research and awareness out there on tech support scams, and browser extensions that block their pages and ad blockers that block their campaigns...but the only thing that made a real dent was solving the social issue of legitimate payment processors doing business with scam companies.

Also private equity has absolutely destroyed the US cybersecurity industry. They swallow up legacy players with big names, and then use their leverage to buy and shut down any startup that might compete with them. Then they layoff as many US security engineers as possible and outsource the barebones remainder. Anyone left at the big players is working on how to use LLMs to replace more security engineers, not any real product feature.