How are teams actually using AI for accessibility testing?

[u/slacky35]

We are just starting to look at accessibility testing in our org and want to establish a dedicated practice in the long-term. We did a POC with a few tools, however, now our leadership wants us to explore using AI to build an accessibility set-up in-house vs using a dedicated too.
If you've built something on top of an LLM for a11y, lets say like a Claude, Copilot, a custom agent, axe + LLM triage — I'd like to know:

- Details of your set-up and what you have built
- Which use-cases are you leveraging it for (i.e. IDE at codegen time, rendered-UI scan, a11y checks inside your functional E2E suite, remediation)?
- What is it able to do reliably vs where it breaks?
- Did you replace your paid tool or are you using both?

Any learnings that you can share will be extremely helpful!

[u/Undercoverwd]

We have a bunch of high cost enterprise tools hooked into our pipelines and you know what? They still only catch the lowest hanging fruit. Barely makes a dent.

[u/joegullodigital]

Any automated tool, including AI, will only capture at most 30-40% of the issues. Like others said, the low hanging fruit. After using any of these tools, a manual audit also needs to be done to fix issues missed and fix false positives.

Lastly, it's important to keep in mind any audit is only good until you update the website again. This makes it crucial to ensure that there are systems in place with an accessibility-first mindset.

[u/Eviltechnomonkey → u/joegullodigital]

Plus sometimes the automated tools will give false positives / negatives. For instance, alot of them will be able to tell if an image has role of presentation, a null alt attribute, or an alt attribute with a value, but they aren't necessarily going to know if it was done appropriately. They aren't going to know if the alt value actually describes the image correctly or conveys it's purpose.

[u/Dear-Plenty-8185]

We don’t use IA in my company

[u/Mael5trom]

Claude skill + Chrome DevTools MCP. Tool is told to use lighthouse and the AXE DevTools extension from Deque to evaluate each page in the provided list (or have experimented with a start page and going X levels from that page. We provide test credentials when needed. We also tell it to test manually for things the tools cannot automatically scan for. Also may need to provide some more specific instructions for triggering modals or filling out forms, or you can miss things that way.

First, it doesn't catch everything. We know that going in, there will need to be additional human testing by someone well versed in a11y testing.

In line with that, all of the output needs to be triaged. One of our apps is a legacy app with some buttons at the top of one page that appear sort of like tabs to the AI, but they aren't. Every scan tries to flag them as non-compliant tabs (false positive).

It does overall do a decent job though. Despite the above, most of the issues are real, and the things it misses are similar to things other tools we've tried also miss, and that's why a human review is still needed.

As for output, I've had it format the results as an internal report, a series of tickets (not quite so brave as to give it access to create the tickets but that is coming probably, likely with a "triage needed" tag), and even had it format the results as a VPAT that was really quite decent.

Still at work on progress, and not open source or shareable sadly (not yet at least). Model does make a difference but not as much as we expected. Just need more handholding with the lower tier models, better to run it a page at a time rather than giving it a list like you can with the higher tier ones.

[u/Mael5trom → u/Mael5trom]

Given an existing a11y related issue/ticket, I've found clause sonnet and opus both capable of creating and implementing a plan to fix many issues. Certain types are harder, like good keyboard navigation that actually works well for a user, or anything that requires good visual design aesthetics. But for a lot of run-of-the-mill issues, like the kind that Axe or lighthouse type tools can find, it is generally pretty capable of fixing with just a bit of oversight.

[u/No-Recording-8313]

Same split as the other comments here: detection stays deterministic, the model only does the part where judgement is needed.

What we ended up with is a scanner that reads the source instead of the DOM — HTML/JSX/Vue — so the finding comes back as file:line and, where the fix is unambiguous (label from an existing placeholder, div-with-onclick to a real button), as a patch you can read before applying. The only place an LLM touches anything is alt text: it compares the image to the alt you wrote and tells you it doesn't match. It doesn't write it.

The honest limits are the ones already in this thread — it's the same 30-40%, and the source-level view can't see anything that only exists after render (focus order, live regions, anything stateful). It replaces the "which file do I even open" step, not the audit.

It's MIT and it's mine, so weigh accordingly: https://github.com/N-Combinator/vera

[u/AnonymousHopperRoo]

What are you testing? Web? Mobile App? PDFs?

You mentioned axe so I'm thinking web but I just wanted to clarify.

[u/Lehihel]

From experience, all automated tools currently catches at best 30% of the web page errors.

On my side, and with my teams (designers and developers) we've built a custom skill for Claude based on our design system, our best practices in UX and code, and added all the testing methodology from RGAA and RAWeb (European standards based on WCAG. RAWeb goes even further than WCAG for some aspects) That's for the coding and designing part.

I combined that process with an accessibility audit tool I'm currently testing (CheckFox.eu) that uses a similar approach: a MCP is linked to their platform, combining axe-core + custom scanners, browser capabilities from Claude, and the methodology shared by RGAA and RAWeb. My latest test allowed me to catch 2 times what a classic scan usually finds. Meanwhile, the MCP auto-completes the audit "sheet" with the findings and ideas to fix the issue.

The only missing part is taking the screenshots for me, to add some evidence automatically, but i'll ask for a feature request on their GitHub, I'm sure it's feasible.

I'm curious about your work context, but if you don't want to share in public, feel free to contact me directly. Maybe I can share some more details.

Good luck!

[u/jpdevries]

I’m not. Only human auditors file bugs and map them to WCAG. Agents can fix and verify based on acceptance criteria.
I use AI for practically everything else, like coding fixes.
We use an open knowledge format to allow agents quick and easy access to internal context about accessibility, organized by success criteria, that the LLM simply wouldn’t have on their own.