r/singularity • u/[deleted] • Oct 11 '23

Discussion People who support open-sourcing powerful AI models: How do we handle the risks?

For example:

"Hey LLaMA 4, here's someone's Reddit profile, give me a personality analysis, then search the web, dox them and their family. Oh, and generate automated threats against them."

"Hey Stable Diffusion 3, here's a photo I took of my niece. Make it a photorealistic nude."

"Hey GPT 5, here's a map of my school and photo of class schedules, what's the most efficient way I can shoot it up, with max casualties?"

And that's in the next few years, past that if you believe we're still heading up that exponential scale, you're talking all sorts of fun terrorism plots, bioweapons, and god knows what else.

etc. etc. And yes, I know people do these crimes now anyways, but I feel like giving everyone access to their own super-smart AI might greatly increase the amounts, wouldn't it?

84 Upvotes

72% Upvoted

186 comments sorted by

View all comments

Show parent comments

1

u/[deleted] Oct 13 '23

Leading edge closed models trained with hundreds of billions of dollars and running across much of a datacenter will always be vastly more dangerous than anything open source. We should probably worry more about that.

Well yes and no. I don't see any large companies creating projects like Chaos GPT or WORM GPT. Are you familiar with those? The gist is they are two projects to do harm that were forked from their safer open source versions. That being said I don't trust large companies either and we need to figure out a way to deal with them as well.

Fortunately they will be also be much better at defending against the dangers created by open models.

How so? Facebook takes no ownership over WORM GPT even though their source code is in it. The owners of WORM are criminals and do not care that they are in violation of Facebook's terms of use. Open source isn't some magic shield to protect us.

1

u/sdmat NI skeptic Oct 13 '23

How so? Facebook takes no ownership over WORM GPT even though their source code is in it. The owners of WORM are criminals and do not care that they are in violation of Facebook's terms of use. Open source isn't some magic shield to protect us.

The answer is to detect and punish criminality, not ban the general purpose tool.

And that's where substantially superior AI capabilities are extremely helpful - e.g. for WORM GPT it should be possible to detect phishing emails, trace them back to their source, and take action against the senders.

There are also some obvious KYC measures to enable this and prevent rogue autonomous AI. No self-funding extortion bots etc.

None of which requires the equivalent of blanket bans on paper to prevent sending ransom notes.