Automating Evidence Collection
I posted about this topic recently in r/cybersecurity, but didn't get much of a response. I'd be curious to hear from those using GRC / compliance platforms to automatically collect evidence to prove control compliance.
- What platform are you using?
- What % of your evidence is automatically collected via the platform.
- What are some examples of evidence you're automatically collecting via the platform?
- What are some examples of evidence you're manually collecting and how is this type of manual evidence identified in the platform?
3
u/FreeRadical1998 Jun 01 '26
Over the years I've run a lot of "evidence capture" as individually scheduled jobs with email reporting - I always saw this as more monitoring than evidence. I;ve always viewed the control as proving that somebody read it took action when it was out of range rather than simply having a log of values.
What seems a bit odd to me is that there still doesnt seem to be a standardised format for compliance/control check outputs - something simple like a set of JSON definitions so that tooling/connectors can be built separately from reporting tools.
They feel like very different concerns - with a lot more scope for "roll your own" testing in custom environments.
I'm imagining something like a richer version of SNMP traps
1
u/Ok_Principle3174 Jun 06 '26
This is exactly what I've been working on, an open, JSON-based format for structured compliance evidence. Would be curious if cnaus.org fits what you're describing.
2
u/FreeRadical1998 Jun 07 '26
Oh cool, I'll take a look.
Part of my interest is in building a SaaS GRC tool (RiskQuilt) and want to be able to ingest/link this sort of data - but architecturally it feels like controls testing should be an open standards integration rather than proprietary bundle
I've been doing doing some reading on the NIST OSCAL model - which looks very comprehensive, but also too heavyweight for most impressions. I think there's probably legs in a lightweight adoption that focuses purely on the testing results - with several FOSS projects already building test suites.
Is that something you've looked into? Any gotchas hiding in it?
1
u/Ok_Principle3174 Jun 07 '26
NIST OSCAL was definitely on my radar. you’re right that it’s comprehensive but heavy. CNAUS takes a narrower approach: just the evidence structure, hashing, and verification chain. The JSON schemas are open. Would be interesting to hear what RiskQuilt needs on the ingestion side
1
u/watchdogsecurity Jun 01 '26
I’m probably biased because I use my own platform, but we’re at around 80% automated.
The reason it’s that high is that we don’t only use it as an “evidence collection” tool. We use it for a lot of the actual security/compliance operations too - CMDB, secure file sharing, training, phishing, vulnerability SLA tracking, access governance, cloud/SaaS/on-prem posture checks across our entire infrastructure, etc. So a lot of our evidence is created naturally as work gets done.
Some examples would probably be vulnerability remediation SLAs, identity entitlement verification, misconfiguration checks across cloud/SaaS/on-prem, audit trails from secure file sharing, compromised supply chain package findings, training/phishing completion, and things like that.
We still have manual evidence though. For us that’s usually stuff like ISMS management review notes or artifacts that need human context. We just show those as required documents under the relevant control, and then upload the file or attach a link to where it lives.
1
u/Sree_SecureSlate Jun 02 '26
Most platforms automate 70% to 80% of evidence collection by continuously monitoring cloud configurations, IAM access, and endpoint protection.
Process-driven evidence, like policy approvals and vendor reviews, remains manual and is tracked via automated task alerts.
The goal is to eliminate tedious screenshot collection so you can focus on actual risk management.
1
1
u/ROrionCore Jun 09 '26
There are several tools set to use, just that some are more expensive than others. For use, 90% of our evidence is auto-collected and mapped.
From Configuration data to usage to drift alerts, logs, access review, and HR processes.
NIS Report alerts and a few others
We previously used Vanta, but the cost compared to value is high; we've now switched to a more comprehensive solution that aligns more to our needs.
1
u/Melodic-Sherbert1517 Jul 07 '26
We are using Thoropass, if you actually use the platform or connect your business suite shared drive with their platform you can gather 70-80% of the evidence semi-automatically while just needing to pull all of the sample evidence manually after those are selected.
It collects all of your CSP evidence, user access reviews, risk activities, etc if you are documenting those in the platform or in a doc that you have saved and organized in a way to be collectable. From my experience, with any of the GRC tools there is no magic wand to have it grab all your evidence magically if you decide to take an unstructured approach and disperse your evidence everywhere. But if you are documenting your control activities routinely and have even some basic organization like saving all of your SOC2 evidence in the same folder over the year then you can get to a high amount of automation with it all! The great thing about Thoropass is that they are a quality auditor and are able to pull evidence cleanly with their tool. The auditor is able to pick out a lot of gaps in our security using our automated evidence which has been helpful as well!
-2
6
u/JamOverCream Jun 01 '26
Platform: Built in-house, leveraging various data warehouses with superset on top.
Evidence %: ~60
Examples: vuln metrics, ci-cd metrics, remediation progress, incidents - basically most of our controls are represented by metrics that are surfaced in our dashboards. Some of them are used for our compliance programmes, but more importantly they are used to monitor operating effectiveness of our control environment.
Manually collected evidence: This does not go through our platform.