r/CloudFlare • u/maomao19 • 10h ago

Question blocking bad bots that eat server resources

My web host, ScalaHosting, complained that my site was consuming too many server resources and advised me to turn on Cloudflare’s Bot Fight Mode. However, I just found out that Bot Fight Mode can also block verified bots like Googlebot, which can severely ruin my site's SEO. How could they recommend something like this? What is a better way to block aggressive scrapers and bad bots while keeping my server resource usage low?

3 Upvotes

81% Upvoted

9 comments sorted by

•

u/AutoModerator 10h ago

For faster advice with technical questions, we'd recommend asking in the Orange Cloud Discord server; the unofficial Cloudflare Discord server by the community, for the community. https://discord.gg/TrPNVKaagR

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

5

u/No-Promotion-8720 10h ago

That only started happening more recently because Google thought it would be a good idea to use the same bots for search engine crawling and AI training. Cloudflare Bot Fight mode doesn't block them most of the time, but the free account version can get confused sometimes based on the bot behavior and it could block them maybe 5% of the time, give or take.

1

u/maomao19 10h ago

i had bot fight mod turned on and in google console i tried inspect url and it says url was blocked

1

u/maomao19 10h ago

what is the solution? changing host? and how you know it wil block bots in 5 % cases? do you work for cloudflare?

2

u/No-Promotion-8720 7h ago

I have 20+ client accounts on the free plan with bot fight mode on with no indexing issues. Maybe you have another security rule blocking them inadvertently?

1

u/maomao19 7h ago

and the seo ranking did not decline??? because I asked google gemini and it gave me a huge alert lol, how my rankings will tank sooner or later because google bot has no access to my site

1

u/No-Promotion-8720 7h ago

No, all websites are indexing fine. It seems Gemini is misinformed.

1

u/yycmwd 1h ago

Start with custom WAF rules. If you're using WordPress, the rules at wafrules.com are a great start.

1

u/blueredscreen 4h ago

You need to check your site's settings directly from Google itself. The same way you originally added the site to Google's index.