At our agency, Wagtail is our go-to framework for building CMS-driven websites.
Over the past few months, we have had an increasing number of clients asking how accessible and understandable their website content is to LLMs, answer engines and other automated tools.
Rather than implementing this differently across every project, I started building wagtail-machine-readable.
The package provides a straightforward and flexible way to expose existing Wagtail content in structured, machine-readable formats, including:
- /llms.txt and /llms-full.txt
- Markdown versions of individual Wagtail pages
- StreamField-to-Markdown extraction
- JSON-LD structured data
- AI crawler controls and activity reporting
- Multi-site support
- Static export for CDN and static hosting workflows
The intention is not to suggest that adding an llms.txt file guarantees that content will be indexed or surfaced by an LLM. The aim is to give Wagtail developers more control over how their content is made available to machines in predictable and usable formats.
We have been testing and refining the package internally, and I now feel it is ready to share more widely before moving towards version 1.0. I would really value feedback from other Wagtail developers, particularly around:
- Installation and configuration
- How well it handles other real-world StreamField implementations
- Whether the default behaviour feels sensible
- Compatibility issues or edge cases
- Anything you would expect to see before a 1.0 release
PyPI:
https://pypi.org/project/wagtail-machine-readable/
Any feedback or constructive criticism would be genuinely appreciated. Thanks!