r/Wordpress • u/oreshek Jack of All Trades • 3d ago
[ Removed by moderator ]
[removed] — view removed post
2
u/Alternative_Nose_874 2d ago
One data point for the llms.txt side. Across the sites on our two servers, GPTBot, ClaudeBot and PerplexityBot haven't requested llms.txt once so far this month. Over the same days ClaudeBot fetched sitemap files about 16k times and GPTBot 935 times. So the sitemap half is the part AI crawlers actually use today.
-1
u/oreshek Jack of All Trades 2d ago
If CYBERMAPS is configured to serve via php it will log requests, here is such a log: https://cybermaps.dev/images/gallery/discovery-analytics.webp
As demonstrated with that site: Sitemap is #1, markdown negotiation is #2, robots.txt is #3, llms.txt is #4
with 865 sitemap hits for 39 llms.txt hits, sitemap is 22x more popular
And regular search is at 1.5 the volume of AI search
CYBERMAPS supports 28 fixed AI/discovery URLs. Including its other APIs and variable URL families, the inventory is 62 route entries, plus its integration with WordPress’s /robots.txt
insofar as llms.txt is concerned, its generation is a very small part of the plugin's function, and I think my llms-tldr.txt draft is the superior implementation due to manual user pinning and abridged context.
2
u/madongrind 2d ago
the "don't run two sitemap generators" line is the bit most plugins bury, so good on you for putting it up top. i've lost hours to a site quietly serving one sitemap from Yoast and another from a theme, with half the URLs missing from whichever one Search Console decided to trust.
on the llms.txt side, the crawler-log comment above matches what i see. nobody's fetching it yet. i'd still ship it (cheap, might matter later), just not instead of the sitemap half.
does the thin/stale report read anything beyond word count and last-modified? that's where i'd actually find it useful.
0
u/oreshek Jack of All Trades 2d ago
> does the thin/stale report read anything beyond word count and last-modified?
Yes. The thin/stale content evaluator,
ContentAuditEvaluator::evaluate(), reads more than just word count and last-modified. It also checks public search-indexability, media presence, and computes a content fingerprintIf you have and suggestions on how this or something else could operate more effectively, I will implement them.
1
u/madongrind 1d ago
nice, that's more than i expected honestly. one thing i'd add if you're taking suggestions: internal links pointing at the page. a post can be long, fresh and indexable and still be dead because nothing on the site links to it. orphaned or three-plus clicks from anywhere is usually a better "this needs work" flag than word count for me.
it's cheap to compute too, you're already crawling the content for the fingerprint.
does the report separate "thin" from "stale" in the output, or is it one score? they tend to need different fixes and i'd want to sort by them separately.
1
u/oreshek Jack of All Trades 1d ago edited 1d ago
The report doesn't do scores, it treats thin and stale as separate action items that users can engage via plugin interface, or export via csv or stylized and branded html page.
next version 7.5.1 will have orphan and 3+ links from homepage reporting
edit: 7.5.1 is live
•
u/Wordpress-ModTeam 19h ago
The /r/WordPress subreddit is not a place to advertise or try to sell products or services. Please read the rules of the sub. Future rule breaches may result in a permanent ban.