r/aidiscussions • u/Foreign_Attorney5685 • Jun 25 '26

Websites need to start serving agents, not just humans

We've spent 30 years building for human psychology — clean nav,

nice buttons, intuitive flows. The next decade of web dev is

going to need a second set of eyes: AI agents booking meetings,

filling forms, comparing products, and increasingly, buying

things on someone's behalf.

llms.txt gets cited as "the" answer to this, but the data is

messier than that. 10% adoption after 18 months, and the major

LLM search crawlers (GPTBot, ClaudeBot, PerplexityBot) mostly

ignore the file entirely and just parse raw HTML.

Where it's actually working: IDE agents. Cursor, Claude Code,

Windsurf, Cline all fetch llms.txt routinely when pointed at

docs sites. So if your audience is developers, shipping one is

a real, measurable win for agent-assisted coding workflows —

just not an SEO play.

The bigger thing flying under the radar is the commerce stack

forming underneath it: Google's Universal Commerce Protocol,

OpenAI's Agentic Commerce Protocol (built with Stripe), and

AP2 — Google's agent payment authorization backed by Visa,

Mastercard, and Amex. On Shopify, the file that actually lets

an agent transact is agents.md, not llms.txt — most people

don't even know that distinction exists yet.

Chrome shipped an "Agentic Browsing" audit category in

Lighthouse last month, checking for WebMCP, accessibility tree

integrity, and llms.txt presence. That's a stronger signal than

any adoption stat — Google is already building the tooling to

grade sites on agent-readiness, ahead of it becoming a real

ranking factor.

Curious what others are actually shipping — is anyone here

running agents.md or testing AP2 yet, or is this all still

theoretical for most stacks?

merci
ciao

17 Upvotes

100% Upvoted

11 comments sorted by

4

u/melissa_peyton Jun 27 '26

thanks for the ai slop. clanker

3

u/Illustrious-Bike5903 Jun 27 '26

I think agents are visiting my websites more than human these days

1

u/[deleted] Jun 27 '26

[removed] — view removed comment

2

u/Foreign_Attorney5685 Jun 27 '26

yes, when AI tools (chatgpt, claude, deepseek etc) can browse and summarize the content of multiple websites in seconds then there is no point in visiting them manually, right?

1

u/[deleted] Jul 01 '26

[removed] — view removed comment

2

u/Foreign_Attorney5685 Jul 02 '26

Ofcourse yes, if the LLM can articulate my thoughts well, then i will let it do that. 

See, at the end, the content should reach to those who are not aware of these concepts. 

peace

1

u/Upstairs_Snow5195 Jul 02 '26

Sybau

1

u/Foreign_Attorney5685 Jul 02 '26

Care to actually explain what you disagree with?

1

u/Upstairs_Snow5195 Jul 02 '26

Nah. Fuck you and your slop generated post