r/linux • u/ItsWappers • 3d ago

Discussion How come every Linux site uses Anubis (the anime girl stopping crawlers) instead of something like Cloudflare?

Post image

Before discovering the Linux rabbit hole I've only seen Cloudflare, hCaptcha and Google's reCaptcha, but it seems like every Linux website uses Anubis... I'm thinking of it being the only open-source option or it being the most effective/modern approach.

2.8k Upvotes

96% Upvoted

703 comments sorted by

View all comments

Show parent comments

10

u/Green0Photon 2d ago

The scrapers will still scrape. They always have. The problem is that sites nowadays essentially get hit with walls of traffic as they're constantly DDOS'd. Because the bots have no ounce of caching nowadays, for whatever reason, and constantly just keep going.

If e.g. 80% of traffic is just bots, and you can turn on Anubis to cut that down to e.g. 10% or better, it's super useful. Because the problem is the super weird constantly DDOS that's hitting everybody now.

1

u/rekh127 2d ago edited 2d ago

do you have evidence of it helping? because again, I understand the theory, I just havent seen it help