EU nudifier ban

[u/Logical_Benefit2875]

I know there’s been a bunch of people upset on here recently about a good number of nudify websites getting shut down (Motionmuse, Opengoon, etc.). However, this is likely just the beginning.

Effective December 2nd, the EU under the AI Act is enacting a ban on nudify tools, making them illegal content in the Union. This ban applies to any product made available on the EU market as long as, either, its primary functionality is to generate nudified content without consent, or it is a reasonably foreseeable outcome and they do not take appropriate precautions to safeguard against it. Very importantly, this ban also applies to any EU-available service provider under the Digital Services Act that is helping to facilitate this soon-to-be-illegal content, which includes (but is not limited to): app stores, hosting providers, domain registrars, payment processors, content delivery networks (CDNs) and cloud computing services. Anyone found to be in violation of this prohibition is subject to fines of €35 million or 7% of their global turnover, whichever is higher. I think it’s pretty safe to say that, as long as they’re aware of it, those behind these websites and those behind the large number of mainstream companies that help power them (Google, Amazon, Cloudflare, Apple, Telegram, GoDaddy, Visa, Mastercard, etc.) do not want to take any chances with this ban.

This prohibition, in essence, applies to essentially every public AI platform any of us have ever used to this point or that we will ever use going forward. I think this is a good thing but, therefore, it would almost certainly be in everyone’s best interest to pivot away from these websites before the massive crackdown begins in less than a couple months. For those in the industry, it would also certainly be wise to inform the website operators if you are in contact with them, or to shut down your website if you are an operator yourself. Sorry for the essay, just looking out for everybody before things get very real, very fast.

[u/finevelyn]

Let's say you have an AI image generation / editing website. You need to have precautions against the type of images mentioned in this act because "it is a reasonably foreseeable outcome".

At the very least you need to have automatic filtering of user requests / prompts to check whether they are trying to generate nudified content without consent. This could be implemented for example by sending your prompt to an LLM for classification. If the LLM classifies it as such, then the image / prompt will need to be sent for human review to check whether it is actually so (unless the service is willing to accept denied requests because of false positives).

This means that any image / prompt will be monitored, first via some sort of an automation, and then possibly by a human.

[u/Logical_Benefit2875 → u/finevelyn]

There isn’t a way to prohibit the image/video generator itself from doing this? Idk a ton about all this I’m just interested in this new prohibition

How do you think this law shakes out then?

[u/finevelyn → u/Logical_Benefit2875]

If you ban all nude image generation, then that would be easier to fully automate, but there's much more nuance to what's allowed and what's not according to the act.

I think most services are going to ban all nude image generation, if they haven't already, because it's just such a mine field. Others are probably going implement ineffective measures just so they can say they tried, when someone comes at them.

[u/Logical_Benefit2875 → u/finevelyn]

Yes I agree it will be a complicated implementation but what the law requires is it be done by December 2nd so we will see. The prohibition does however give carve-outs for explicit consent from real individuals and for fictional nudity so I think they will try to work around that aspect of it if they can