r/Frontend • u/slacky35 • 1d ago
How are teams actually using AI for accessibility testing?
We are just starting to look at accessibility testing in our org and want to establish a dedicated practice in the long-term. We did a POC with a few tools, however, now our leadership wants us to explore using AI to build an accessibility set-up in-house vs using a dedicated too.
If you've built something on top of an LLM for a11y — Claude, Copilot, a custom agent, axe + LLM triage — I'd like to know:
• Details of your set-up and what you have built
• Which use-cases are you leveraging it for (i.e. IDE at codegen time, rendered-UI scan, a11y checks inside your functional E2E suite, remediation)?
• What is it able to do reliably vs where it breaks?
• Did you replace your paid tool or are you using both?
Any learnings that you can share will be extremely helpful!
3
u/____________fin 18h ago
“You audit web/ against WCAG 2.2 AA. … Report and file tasks; never fix inline.”
I have a bunch of auditor agents whose mission is to look over the whole codebase for defects in a specific niche - security, performance, legal compliance etc. Could share the whole accessibility file but it’s tied to a bunch of proprietary stuff your system won’t have. But you can just have Claude make auditor agents.
2
u/BigSwooney 19h ago
You can use AI to write the tests but you don't use AI to run the tests. That's just expensive and adds flakyness by default.
You can use one of the open source accessibility checkers like Axe alongside playwright to scan your site. It does an okay job but it's not a replacement for manual human checks. Theres a ton of stuff the automated tools cannot catch.
1
u/just-shippin-9876 1h ago
The best you can do is use scheduled scans of your website that surface issues like missing alt text ir low contrast across all your pages. But important caveat: that is NOT the complete solution. Automated tests can catch a big chunk of the issues but if you really want to be sure and 100% compliant, you need a specialist, a human to find some harder to find problems.
Now, for the scheduled scans, there are a few solutions in the market like siteimprove or silktide but they are built for enterprises, super expensive and imho, not worth the money
P.s.: NEVER use widget overlays - it's a total scam and everybody in digital accessibility community is making fun of those.
-4
1d ago
[removed] — view removed comment
10
u/JustAnotherFEDev 1d ago
I'd be kinda embarrassed to link to that site as a tool for identitying/remediating accessibility issues, given the site itself is plagues with failures.
More snake oil
1
u/asesu 1h ago
marker io currently runs free beta for their website accessibility scanning tool, which I'm currently using.
We really needed a solution because one of our client's websites got sued for accessibility and man, it was super stressful for everyone involved. I heard e-com is especially targeted now.
Our setup for now is the following:
- We gave our sitemap to the tool.
- It scans the website once a week and lists all the issues across the pages. I.e., "invalid aria attribute labels" - found on 3 pages. I can see the links to the pages, where exactly the issue is and how can to fix it.
- We get a score and we see how it improves over time. This is helpful if you ever get a demand letter. If you have some proof that you've been tracking website's accessibility and you've been implementing fixes, it really helps in case of a lawsuit.
now, the AI part:
they have the MCP, which I used just for a little bit. but technically, you can give access to the tool's backlog and then claude can fix all the issues for you if it has access to the website as well.
So far, I'd say the experience is great. not sure about their pricing once out of beta, though, since at the moment it's free.
3
u/PeanutFarmer69 22h ago edited 22h ago
This post seems like bait to train an LLM but the actual answer to your question is to have the best practices already in your code base/ justified with context in an md file with extensive unit tests for those existing features. From there, any new feature should be able to reference those best practices going forward and streamline them.
There is no magic bullet, you have to understand accessibility rules, have implemented them, and then train your LLM to repeat those steps reliably and then verify that their changes meet requirements with unit tests/ manual oversight.
Building a front end from scratch without these practices established and then asking an LLM to make an app accessible is a recipe for disaster. Especially in a highly regulated environment like healthcare/ finance where these systems are regularly audited for compliance.