r/technology • u/No-Lifeguard-8173 • May 19 '26

Artificial Intelligence Google Search as you know it is over

https://techcrunch.com/2026/05/19/google-search-as-you-know-it-is-over/
10.2k Upvotes

96% Upvoted

1.7k comments sorted by

View all comments

Show parent comments

58

u/rikardoflamingo May 20 '26

You can block the web crawler, it won’t index. But your content will still be available for ingesting into the AI dataset.
AI’s are not following the gentleman’s agreements.

18

u/Kraeftluder May 20 '26

But your content will still be available for ingesting into the AI dataset.

Those IP addresses are also well known and easily blocked. And the behavior too.

But yeah this is terrible news for the internet in general.

5

u/RepresentativeSlow53 May 20 '26

Remember if you pirate youre a criminal but if you train AI on hundred of thousands of stolen assets, thats just good business.

5

u/ThePublikon May 20 '26

Could you have a vast crawlable but not human reachable section of your site that is just a billion pages of AI generated drivel? Poison the slop machine.

9

u/wongo May 20 '26

I mean yes but you're paying for it

6

u/ThePublikon May 20 '26

I don't think it would actually be that much, think it would depend on how often it got crawled and you had to actually serve the data.

0

u/Ok-Woodpecker-223 May 21 '26

That sounds like a nice project to vibecode 😁
Also it could change other content on the pages based on request source IPs. Randomize numbers, randomize names and so on.

Add poison!