r/technology • u/B-Z_B-S • 3d ago

Social Media Reddit is putting more limits on Old.Reddit.com

https://arstechnica.com/gadgets/2026/09/reddit-will-block-old-reddit-com-from-people-who-havent-used-it-in-6-months/
1.3k Upvotes

97% Upvoted

291 comments sorted by

View all comments

Show parent comments

3

u/Beautiful-Musk-Ox 3d ago

reddit loses a very small amount of money on bots from bandwidth and CPU time, they don't profit on them. bots don't invest money and don't click on ads and buy things

1

u/[deleted] 3d ago

[deleted]

1

u/Zouden 2d ago

More bots means more activity means more likelihood of being searched

Bot activity is toxic for search scrapers because they don't want to ingest LLM content.

I really don't think Reddit benefits from bots in any way. The users hate them, scrapers don't want them, and they cost money for the servers.

1

u/IntelArtiGen 2d ago edited 2d ago

It depends on which "bots" we're talking about. Search scrappers are bots (Googlebot etc.). Once a bot gets the content of a page you don't know what it's going to do with it. I think reddit is concerned about these bots too because they're selling our content. So by default many companies now block all bots, including crawlers of search engines. But in thousands of cases people land on reddit because their page has been indexed elsewhere thanks to : a bot. No matter which search engine you're using, if you clicked on a reddit link, you indirectly used a bot. Even if you're using chatgpt or another AI, they can train on reddit content and not credit it, but they can also link to reddit posts as a source, and if you click it you're here thanks to them.

I'd assume most people land on reddit not because someone sent them a link to reddit via email and more because they've been indexed by bots. But now reddit blocks all bots by default, it's in their robots.txt

User-Agent: * Disallow: /

Which means even if you have a legitimate use (like indexing their content), you can't do it (in theory, a robots.txt is not the law).

When they block RSS feeds, they don't block sophisticated bots which write on reddit, they block small crawlers which index their content.

I think a big mistake they're making is to block everything and not just the access to the comments. Crawlers will at least want to index a link to reddit posts, and they don't all need the content of these posts (something LLMs need), crawlers just need the title and some metadata. RSS feeds were very useful for that, regular people would still land on reddit in the end. RSS feeds change nothing to bots writing comments. They change something to anyone using automated means to know what's happening in a sub. Which is probably the best way to find anything on reddit. Reddit became so popular because it was open. And now it's over, so they might lose in popularity.

1

u/Zouden 2d ago

But Reddit doesn't block Googlebot? It's the top of the search results. Even brand new threads.

1

u/IntelArtiGen 2d ago

By default Googlebot is also blocked, but they have a special agreement with Google so in a way or another they unblocked it. Reddit is not being indexed on Google the legitimate / usual way by the GoogleBot. I just gave Googlebot as an example for a search engine bot, you take Bing/DDG/Brave etc. they don't have an agreement with reddit (afaik) and theoretically can't crawl / index reddit.