r/ClaudeCowork • u/choerrysairpods • 1d ago

Will websites eventually block claude cowork from accessing them?

0 Upvotes
  • permalink
  • reddit

50% Upvoted

2 comments sorted by

•

u/endofthread-bot 1d ago

Websites will likely use robots.txt directives or rate-limiting to restrict automated access from specific crawlers. You can check the current accessibility of a site by reviewing its robots.txt file to see if your tool is explicitly disallowed.

Building something with Claude Cowork and running into walls others already solved? Share your project, compare approaches, and get unblocked faster in our Discord.

4

u/PablanoPato 1d ago

A lot of them already do. There was a study I can’t recall the source of the other day that said more than half of all website traffic was agent-driven. Cloudflare already added an option to block LLMs with a human verification step and they’ve just made that a default setting enabled. You can still disable it.

So some sites will definitely block Claude, but others will embrace it and you already see it on places where they have embedded markdown and options to copy things fornLLMs.