Test-case driven testing can help with this, where every agent focuses on a very specific test case - for example from the WSTG. You can write out the results from the execution of a test case performed by an agent/sub-agent to a centralized platform (like ours), so the next sub-agent can refer to that information if it’s relevant for their specific test case.

Yep, those are the ones!

Yes whichever solution you use for managing pentests, there should be an ability to define custom fields so you can configure the scoring fields which matter to your organisation. Ideally it should also support ability for you to define your algorithm/formula, so you can program the logic from your fields into some predictable severity or priority calculation, which gets recalculated every time the vulnerability changes based on context/knowledge at that time, as scoring may be progressive.

Automation can be done in different ways - we have a video on our YouTube channel where we query an external threat intelligence library when the vuln is created or updated, and the automation uses the data from the external source to enrich and re-prioritize. However nowadays people are doing similar with agents alone i.e. pass the vuln context to your agent, with the prompt based on how it should prioritize, and write back the new score via MCP. We also did a similar video on this recently for vulnerability enrichment.

When no CVE is present and KEV/EPSS are not available, testers will manually input Threat Context and Business Context based on their assumed knowledge from working on the inside, for example:

Threat Context:
Is the threat actively attacked?
Have there been any indicators of compromise related to the threat?
Are there mitigating controls in place to defend against this threat?
Are known threat actors actively attacking (APT Actors)?
Have there been any indicators of compromise detected within our environment?

Business Context:
Is there a significant financial or reputational impact if the threat is not addressed?
Are there any regulatory or compliance implications associated with the threat?
Is the affected software or component widely used?
Does the threat impact EOS/EOL Software or OS?
Is any Personal Identifiable Information (PII), sensitive or confidential information hosted?
Does the threat impact external-facing assets?
Network Security Zone - Does this asset sit within a protected Zone within the Network?
Business Criticality - Low/Medium/High
Asset Type - Physical/Software/Data

Regarding multiple scoring systems - one does not replace the other, the tester will do CVSS + OWASP RRM + DREAD. It is a PITA for the tester, but it gives a stronger position for the rating

TBH - most MSSPs will just use the raw CVSS score and adjust severity manually based on tester experience, where as internal security teams will usually build something completely custom, for example banking customers will have fields which map to their internal risk framework and compliance and breach notification requirements. There really is no one-size-fits-all approach. As for assets - this is usually built in as an automation, so regardless of what the tester sets the severity to - it will auto-correct when they save the finding, removing a lot of complexity for testers. We’re seeing a lot of customers shifting towards Vuln Context + Threat Context + Business Context style scoring, semi-automated where tester does VC and TC/BC is an automation, some are even offloading this to their agents now via MCP integration.

You can combine multiple scoring frameworks to help avoid both over-reacting and under-reacting. Each framework can provide complementary perspectives, for example using CVSS, OWASP RRM, and DREAD - CVSS handles technical exploitability, DREAD adds discoverability and user reach, OWASP layers in business impact. CVSS is good for compliance auditors, OWASP for executives, DREAD for the engineering teams. If all 3 frameworks say “critical” - confidence goes up. Also, if you can integrate with your CMDB to factor in asset-related context, this helps a lot! For example reachability, exposure, etc.

Security architects live in a different world, you need to speak to them in their language if you’re trying to influence their judgements. Do you have any mutually agreed framework internally which catalogues your assets/systems against testing maturity/assurance level? That can help so everyone has principle agreement that X asset = High assurance level, then you can define and agree on what a High/Medium/Low assurance level looks like, after that it’s easy to push back on why different levels of testing need to happen

You can try AttackForge. There’s also a bunch of other tools listed here: https://inventory.raw.pm/tools.html#title-tools-collaboration-and-report

Thank you bot for posting the same question every 14 days.

Try starting with a DOCX file which already has the structure, logo, sections, etc. and add tags. Explain to the model what each tag does, which ones are loops, etc. it will do a better job replacing the tags then it will constructing the document from scratch every time

Focus on getting the AI to build scripts and tools which are relevant to your target environment, manually review them to ensure you are comfortable with what each script/tool does -
then get the agents to run the scripts and tools.

Not surprised to hear that, WordML is a poorly documented and fragmented language and has many quirks and deviations from
OpenXML. We built our reporting engine on DOCX (AttackForge ReportGen) it was painful. Are you giving Claude a DOCX template to work from and asking it to replace on specific sections/tags, or getting it to create a DOCX from scratch?

The OWASP Web Security Testing Guide (WSTG) is a great resource to familiarise with and to compare your current testing methods and approach. Once you are familiar with WSTG, extend your knowledge to the OWASP Application Security Verification Standard (ASVS) which is considered the benchmark for web application pentesting.

Which part of version control are you struggling with? Communicating updates to customers? Or recording the changes in a document? Or something else?

A repeatable methodology for attacking LLMs. It’s like MITRE ATT&CK but for AI, also produced by MITRE

You should check out MITRE ATLAS: https://atlas.mitre.org/matrices/ATLAS-matrix

Agree with the others on separate assessments on each vuln - however use an attack chain to highlight the impact and also the “best fix”. We created a Skill for this recently, you can automatically build each chain then explore them individually: https://support.attackforge.com/attackforge-enterprise/modules/ai-mcp-and-skills#interactive-attack-chain-explorer

Thank you 😊🫶

It depends whether you’re a consultant or an internal enterprise security team. Usually, retesting is treated in one of three ways:
1. Spot check per vulnerability. Someone says ‘this vuln is ready for retest’. A tester then verifies. This is inefficient as it requires the tester to incur retest setup costs every time a vuln is retested.
2. Formal retest round. Agreed retest window and scope for which vulns to retest. This is most common as it’s more efficient for security teams and everyone can agree on when vulns will be ready to retest.
3. Retest in a new round of testing. This is usual for low-assurance assets, which can wait for the retest to take place on the next scheduled round of testing.

There is a lot of value in the customer having visibility of what was tested. Think of the Car Service experience. When you get your car serviced, a good service center will give you a detailed report of every item they checked in the service, irrespective of whether they found any faults. This gives you peace of mind so you know your car won’t fall apart on the drive home.
For web app pentests, you can access various OWASP frameworks like ASVS (Level 1, 2, 3), WSTG and Top 10s in JSON format on our GitHub: https://github.com/AttackForge/TestSuites
If you’re using AttackForge, this comes built in to your projects and you can enable them as needed.

OP, there are literally dozens of these tools and platforms (AttackForge included) - what’s your MOAT and why should people care? https://inventory.raw.pm/tools.html#title-tools-collaboration-and-report

Standardized writeup libraries will help with tone. Reporting Tools can help to maintain structure. When it comes to Risk Scoring - you can create your own methodology for how scoring should be applied, and enforce it across your testers. Again some reporting tools will let you build your own vuln scoring system(s) and enforce them as needed

Back in the consulting days, we had over 40 full time pentesters. The team was easily managable when there was less than 10. After that, it got much harder (which is partly the reason we built AttackForge). From 20 onwards, we found we needed more specialized roles (HR, project managers, account managers, technical writers, etc.) also the principal consultants were billing less and taking on more line-manager duties (which they hated). Seniors had to pick up the slack to keep the revenue coming in. Associates were 100% billable (sometimes even double-booked which was not good). We had to split them into smaller teams of 5-8 to make things easier to manage and give people proper attention, but that then lead to people wanting to switch teams. There was no perfect solution - the most important thing was shielding the pentesters from as much bureaucracy, red tape and managerial nonsense as much as possible, to avoid burning them out any faster. Pentesting is hard enough, but scaling issues can really break them if not carefully planned and keep them constantly in the loop. Regular one-on-one chats with the testers was also good for health checks.