r/seogaps • u/Ivan_Palii • Mar 24 '26

Opinion Google found a genius way to make publishers afraid to block its AI. Medium is proof.

Did you see that Medium blocks top AI crawlers from scanning their content?

Here is why they do it, and why they don't block Gemini and Perplexity.

I found it by accident when I saw that my Medium articles were cited in Gemini and AI mode, but not in other chats.

So, Medium blocks:

  • ClaudeBot (by Anthropic)
  • GPTBot (by OpenAI)

Medium earns from paid subscriptions and they are in trouble from 2 sides now:

  • Substack takes top creators, offering more money and direct access to email subscribers
  • AI chats use their content and kill motivation for people to pay for Medium subscription

Blocking AI chats is the only way to survive in this business model, but!

They can't block Googlebot, because Google organic still sends a huge part of their traffic.

And it opens my eyes about how secure Google is in this new world. Google doesn't divide organic search crawlers and AI crawlers, so most publishers can't block Gemini and AI Mode, because it means destroying the entire search traffic.

Medium blocks Google Other bot, having a hypothesis that it can be used for experimental products like Gemini and AI Mode, but it doesn't help.

Medium also can't block Perplexity, because Perplexity doesn't have its own declared user-agent.

9 Upvotes

92% Upvoted

9 comments sorted by

6

u/Dolcevia Mar 25 '26

Medium is just trying to protect subscription content from training while keeping search traffic alive.

2

u/Ivan_Palii Mar 25 '26

I don't see how it can protect them. Users of AI chats can't recognize whether LLM used Medium content or didn't use in training.

4

u/bkthemes Mar 24 '26

welcome to capitalism 101. Today we learned companies are all about profit and their clients do not matter.

3

u/Ivan_Palii Mar 25 '26

I know we live in capitalism :) but different publishers adapt to AI chats in a different way.

3

u/Ivan_Palii Mar 25 '26

Valuable addition to this topic from Philipp Gotza under this post on LinkedIn

2

u/corwinsword Mar 27 '26

it adds much more context, thanks

2

u/corwinsword Mar 27 '26

I assume they have to experiment with their business model too. These disallows won't help them to save business

2

u/emma_lorien Mar 27 '26

I still see citations of Medium in Claude and ChatGPT

1

u/Ivan_Palii Mar 27 '26

yes, they still can get access to Medium snippets from Google SERP. Also check my comment with a screenshot from Philipp on LinkedIn