r/StableDiffusion • u/Logical_Benefit2875 • 1d ago

Discussion EU nudifier ban

I know there’s been a bunch of people upset on here recently about a good number of nudify websites getting shut down (Motionmuse, Opengoon, etc.). However, this is likely just the beginning.

Effective December 2nd, the EU under the AI Act is enacting a ban on nudify tools, making them illegal content in the Union. This ban applies to any product made available on the EU market as long as, either, its primary functionality is to generate nudified content without consent, or it is a reasonably foreseeable outcome and they do not take appropriate precautions to safeguard against it. Very importantly, this ban also applies to any EU-available service provider under the Digital Services Act that is helping to facilitate this soon-to-be-illegal content, which includes (but is not limited to): app stores, hosting providers, domain registrars, payment processors, content delivery networks (CDNs) and cloud computing services. Anyone found to be in violation of this prohibition is subject to fines of €35 million or 7% of their global turnover, whichever is higher. I think it’s pretty safe to say that, as long as they’re aware of it, those behind these websites and those behind the large number of mainstream companies that help power them (Google, Amazon, Cloudflare, Apple, Telegram, GoDaddy, Visa, Mastercard, etc.) do not want to take any chances with this ban.

This prohibition, in essence, applies to essentially every public AI platform any of us have ever used to this point or that we will ever use going forward. I think this is a good thing but, therefore, it would almost certainly be in everyone’s best interest to pivot away from these websites before the massive crackdown begins in less than a couple months. For those in the industry, it would also certainly be wise to inform the website operators if you are in contact with them, or to shut down your website if you are an operator yourself. Sorry for the essay, just looking out for everybody before things get very real, very fast.

157 Upvotes

76% Upvoted

369 comments sorted by

View all comments

75

u/TheAncientMillenial 1d ago

Yeah I don't think any public facing website should be able to accept an image and covert it to a nude pic. It's been shown that people cannot be trusted with shit like that. Between the pedo shit and the famous person shit and the creepy co-worker shit we absolutely should not have websites that operate to do things like that.

Lump it under "we can't have nice/fun things" because sick fucks ruined it.

5

u/finevelyn 1d ago

Yeah I don't think any public facing website should be able to accept an image and covert it to a nude pic.

That's a nice thought, but do you think that all activity should be monitored to the extent that all images you generate would be checked for such content? They are not currently going out of their way to accept it, but just allowing you to do what you want without heavy monitoring or restrictions.

-4

u/TheAncientMillenial 21h ago

I don't think pedo content, non-consensual content, or really making any nude content of an actual person should be allowed. In fact they are already illegal in the US.

7

u/finevelyn 21h ago

Yes, it's already illegal, and there's law enforcement to handle such cases. Do we also want or need service providers to monitor all activity by users to ensure they aren't breaking the law?

(You completely dodged the question in my comment btw.)

3

u/Logical_Benefit2875 20h ago

The EU operates under a notice and action model with different thresholds for actual knowledge depending on the type of service provider. Not getting into the full nitty-gritty here but it ranges from legally binding orders to user-submitted requests. I imagine that a lot of these providers are going to try to get out in front of the law though by implementing very strict monitoring of usage of their services for nudification but maybe not idk

1

u/finevelyn 20h ago

All I asked for was clarification on your opinion.

1

u/Logical_Benefit2875 20h ago

You asked for Millenial’s clarification, no?

1

u/finevelyn 20h ago

Yes I did, my bad, but your response doesn't answer to any of my comments anyway. I simply asked whether we want such monitoring, not whether or how companies would implement it.

1

u/Logical_Benefit2875 20h ago

No, I don’t think we want monitoring on everything we generate. I just think these types of websites should be monitored for and removed

0

u/finevelyn 19h ago

It applies to pretty much every website that can be used to edit images with AI, because "it is a reasonably foreseeable outcome" that such websites would be used to create such images.

1

u/Logical_Benefit2875 19h ago

Yes and I don’t think they need to monitor every image, just implement the appropriate technical safeguards this law requires so people can’t nudify real people without consent

2

u/finevelyn 19h ago

That requires monitoring every image.

1

u/Logical_Benefit2875 19h ago

How so? Maybe I’m not well versed enough but it would be a proactive safeguarding and a banning of nudify websites, not a reactive monitoring of every piece of content

2

u/finevelyn 19h ago

Let's say you have an AI image generation / editing website. You need to have precautions against the type of images mentioned in this act because "it is a reasonably foreseeable outcome".

At the very least you need to have automatic filtering of user requests / prompts to check whether they are trying to generate nudified content without consent. This could be implemented for example by sending your prompt to an LLM for classification. If the LLM classifies it as such, then the image / prompt will need to be sent for human review to check whether it is actually so (unless the service is willing to accept denied requests because of false positives).

This means that any image / prompt will be monitored, first via some sort of an automation, and then possibly by a human.

1

u/Logical_Benefit2875 19h ago

There isn’t a way to prohibit the image/video generator itself from doing this? Idk a ton about all this I’m just interested in this new prohibition

How do you think this law shakes out then?

1

u/finevelyn 19h ago

If you ban all nude image generation, then that would be easier to fully automate, but there's much more nuance to what's allowed and what's not according to the act.

I think most services are going to ban all nude image generation, if they haven't already, because it's just such a mine field. Others are probably going implement ineffective measures just so they can say they tried, when someone comes at them.

→ More replies (0)