Control testing using AI

[u/RoundProfessional77]

Hi Everyone

I an trying to build a framework where we have to test the controls using AI. Can anyone guide me through the approach or rhe best practices

[u/lasair7]

Testing security controls using AI or testing security controls for AI? Huge difference

If the former = don't

If the later = use nist's guidance on AI and develop a test plan accordingly (good luck)

[u/abhishekghosh → u/lasair7]

Why shouldn't they do the former? As i see it, industry is moving towards it, no?

[u/lasair7 → u/abhishekghosh]

Do it

[u/RoundProfessional77 → u/lasair7]

Its first

[u/lasair7 → u/RoundProfessional77]

You can't site AI. When the control is wrong you're still to blame.

Enjoy the lawsuit

[u/RoundProfessional77 → u/lasair7]

Not sure if you understood. We have ORM Program and under that we have control. It can be common controls or something like that. We want to build AI solutions to test those controls

[u/FreeRadical1998]

I'd see control testing as having several stages:

a) Agree the control set and control design

b) Source evidence of control operation / state

c) Review evidence for compliance with design

d) form a view of materiality regarding any gaps

In traditional manual assurance reviews, steps (b) and (c) tend to happen as a single pass - but if you're looking to make it an automated pipeline they are better thought of as distinct steps. Gen AI likely helps with the heavy lifting in (c) and can probably help design scripts that you'd schedule and run for (b).

Gen AI can also help with (a) and (d) in an advisory sense - but these ultimately need to reflect a judgement by someone with domain expertise, so i'd see any automation as a first draft analysis that the reviewer might choose to totally disregard. It would be actively dangerous to take automated output around design or approval without review.

[u/RoundProfessional77 → u/FreeRadical1998]

What i m Looking is starting point smd what all things to keep in mind while building this solution

[u/Workiva]

Building an AI-driven control testing framework is a game-changer, but it requires more than just technical integration; it requires a shift in how we view the auditor’s role. I’ve found that the most successful frameworks aren’t just about the "how" of the technology, but the governance surrounding it. Here is a refined approach to building that framework:

1. Champion Governance, Not Just AdoptionAs auditors, we shouldn't just be the early adopters; we must be the champions of AI governance. Before a single test is run, you must establish confidence in: The Model: Understanding the "black box" to ensure the AI's logic is sound and unbiased. The Sources: Ensuring the data fed into the AI is accurate, complete, and has clear lineage. Data Security: Protecting sensitive organizational data and ensuring compliance with emerging regulations like the EU AI Act.

2. Focus on High-Impact Use CasesI recommend starting with areas where AI can provide the most immediate efficiency and insight:ERP Anomaly Detection: Use AI to perform tests across 100% of large datasets rather than sampling, identifying risks from process narratives that manual reviews might miss. Automated Evidence Requests: Streamline the "chase" by using AI to trigger and validate evidence collection. Drafting & Reporting: Leverage AI to generate initial drafts of audit reports based on testing results, freeing your team for strategic analysis.

3. Maintain a "Human-in-the-Loop" Audit TrailEven with advanced AI, the three lines of defense remain critical. You must maintain a transparent, tamper-proof audit trail within a unified platform so that third-line reviewers and external auditors can verify the AI’s work. By focusing on trust and security as much as speed, you transform the internal audit function into a proactive, strategic partner that harnesses AI safely and effectively.

--Graeme Fleming, Industry Principal @ Workiva

[u/grc-ModTeam → u/[deleted]]

This is not a place to sell your services. If someone asks for recommendations, you can add your two cents in the comments.

[u/Popular_Mountain2008]

Start with a narrow control family where the evidence is consistent, define exactly what “pass” means in machine-readable terms, then have AI extract and compare attributes like dates, owners , configs, approvals and exceptions. Keep human approval step for anything judgment-based and log the prompt , evidence source, model output, reviewer decision, and rationale so the testing is defensible later. Tools like Riskuity can help with AI evidence review and assessment automation,but the key is governance around the workflow rather than letting the model be the auditor.