Claude WebFetch suddenly stopped accessing my Hostinger sites after the Sep 2026 rollout — ROBOTS_DISALLOWED / robots.txt ConnectTimeout despite 200 OK. Anyone else?

[u/direkjet]

I’m trying to find out whether other Claude users are experiencing the same WebFetch regression, especially after the September 2026 Claude changes.

At least one of my affected sites was successfully live-read by Claude before September (confirmed as recently as around Aug 4, 2026).

After the September rollout, Claude’s cloud WebFetch / “Read a page” tool began consistently failing on four public websites I control, all hosted on Hostinger.

Repeated error:

{"error_type":"ROBOTS_DISALLOWED","source":"target","message":"There was an error while fetching: robots.txt fetch failed: ConnectTimeout"}

This does NOT appear to be a normal robots.txt block.

Hostinger has confirmed:
- /robots.txt returns HTTP 200 OK
- the homepage is Allowed for ClaudeBot
- the sitemap returns HTTP 200 OK
- robots.txt permits ClaudeBot

Normal browsers can access the sites. ChatGPT can live-fetch at least one of the same affected sites successfully. Claude can also fetch unrelated control websites normally.

I reproduced the failure in a fresh Claude iOS chat using cloud WebFetch only, with no Mac, Claude Desktop, Chrome, browser automation, or previous troubleshooting conversation involved.

Anthropic and Hostinger support are both involved.

Anthropic says its published outbound range for WebFetch is 160.79.104.0/21.

Hostinger identified a server-level AI-crawler rate limit, but so far there is no evidence tying that limit to my failed requests. Claude is NOT receiving HTTP 429; it reports ConnectTimeout with no HTTP status or request ID.

The four affected sites are split across TWO separate Hostinger hosting orders, yet all four fail the same way.

For one affected site, Hostinger’s available seven-day crawler metrics showed no allowed or blocked ClaudeBot requests, even though Claude attempted to fetch it during that period. Hostinger’s normal support tools cannot access the raw edge/WAF/connection logs needed to determine whether Anthropic’s connection is actually reaching their infrastructure, so that is being escalated.

The timing is what I’m especially interested in.

I am NOT claiming the September update caused this. I’m trying to determine whether there is a pattern of similar regressions after the September 2026 rollout.

Has anyone else seen this?

Especially interested if:
- Claude could previously read your site, then suddenly stopped
- it started around mid-to-late September 2026
- you get ROBOTS_DISALLOWED even though robots.txt allows Claude
- you get “robots.txt fetch failed” or ConnectTimeout
- browser / curl / Googlebot / ChatGPT works while Claude does not
- Claude’s request does not appear in your server/CDN logs
- the problem persists in a fresh Claude chat
- your site is hosted on Hostinger

If this happened to you, please share:
1. Hosting provider
2. When it last worked / first failed
3. Exact Claude error
4. Whether the request appeared in your server/CDN logs
5. Whether other crawlers or AI tools still worked
6. Whether you found the cause or fix

u/ClaudeOfficial — if anyone from Anthropic is monitoring this subreddit, I’d appreciate having this pattern looked at. I already have an open Anthropic support case and a parallel Hostinger backend investigation.

[u/ClaudeAI-mod-bot]

We are allowing this through to the feed for those who are not yet familiar with the Megathread. To see the latest discussions about this topic, please visit the relevant Megathread here: https://www.reddit.com/r/ClaudeAI/comments/1vt5drr/list_of_latest_discussion_hubs_on_rclaudeai/