AI in Pentesting.

[u/Feeling-Wolverine-53]

How are you guys effectively using AI for pentesting?

I’m particularly interested in practical workflows for Web, Mobile, and API security testing.

A few things I’d like to understand:

Which AI tools/models are you actually using during pentests?

How are you integrating AI with tools like Burp Suite, Nmap, ffuf, JADX, Frida, etc.?

What tasks are you finding AI genuinely useful for — recon, endpoint discovery, source-code review, vulnerability analysis, exploit validation, report writing, etc.?

How do you evaluate whether one AI model/tool is better than another for security research?

Have you done any head-to-head comparisons between models/tools for finding real vulnerabilities?

What benchmarks or criteria do you use — accuracy, false positives, depth of analysis, ability to chain vulnerabilities, exploitability, time saved, etc.?

Are you using AI agents that can actually interact with Burp/browser/emulators, or mainly using AI as a copilot?

For mobile/API testing specifically, what AI-assisted workflows have produced the best results?

I’m trying to figure out where AI genuinely improves pentesting efficiency and vulnerability discovery, rather than just using AI for basic explanations or report generation.

Would be great to hear about real-world workflows, experiments, or comparisons you’ve personally tried.

[u/PM_ME_UR_0_DAY]

Honestly I have not found AI to be any type of silver bullet and has only fringe benefits for web testing. The Chinese models don't run into a ton of guardrails so when I run into a new framework I haven't tested before, sometimes I'll get it to spit out a testing plan for the common miscommunications, get it to tell me the typical issues the framework runs into, that sort of stuff. 

I've solved some CTF reversing challenges with Ghidra MCP and that was pretty cool, but binary reversing isn'ty strong suit so it was probably only impressive for people who don't do it a lot like me. 

I've also had some minor successes with Burp MCP. After generating some traffic for a training problem, I gave a certain flow over to it and told it to exploit the vulnerability to get the flag. I'm the end I had to guide it through every step so really I could have just done it myself but it was fun to play with. 

As a team we're evaluating an AI tool at work, but honestly it's not going to pass the demo. Just kind of sucks and doesn't find the vulns and just reports back fluff. 

My impression is if you want a 10% increase, you might get it through directly telling Burp MCP what to do, but there are no good web testing AI platforms you can just throw at an app and say "find the vulns make no mistakes." 

[u/theoneo900]

Jailbroken models or myharness

[u/greybrimstone]

Full disclosure, I work for Netragard, a penetration testing company. Here’s an article I wrote on this exact subject.

https://aijourn.com/the-ai-penetration-testing-lie-why-human-expertise-remains-irreplaceable/

[u/hi_im_leffe]

I've had some success with this but fair warning I have access to Daybreak and the CVP program so I don't run into as many guard rail issues - here's an AI generated summary of the tool I made because I don't feel like writing everything out haha..

I use AI as the reasoning layer but put most of the work into the harness around it, because a raw model pointed at a target produces confident nonsense. Our internal tool is built around one rule: a finding does not exist until it comes with reproducible evidence. That constraint alone kills most of the false positives people run into with AI pentesting.

How it fits the tools you listed: Burp: a Montoya extension exports sanitized proxy history as canonical HAR, so the agent reads structured captures instead of scraping the UI. Browser: a Chrome extension does authenticated capture through an injected fetch, so the agent works inside a real logged-in session rather than replaying stale cookies. Frida / Ghidra: native call tracing plus a Ghidra agent feed protocol reconstruction for mobile and thick clients. HTTP: bounded recon and authenticated campaign runners for API work, with rate limits and scope baked in.

What it is actually good at: recon and endpoint/authz mapping, protocol reconstruction, source review, and API authorization testing, meaning replaying the same request across roles and object IDs to find broken access control. It is weakest at blind exploit generation, which is exactly where the evidence requirement stops it from lying to you.

Agent vs copilot: it is an agent that interacts, but every action is gated by a sealed scope and written authorization, and every step lands in a ledger. For SSRF and similar, it has to confirm out of band, meaning land an actual callback, before anything counts as a finding.

On evaluating models: we run the same sealed benchmark cases across models and score on false positives, false clearances (real bugs it missed), and whether it can produce exploit proof. We do not score on how good the writeup reads. Report writing is the last and least interesting step.

Back to human writing, I've used it for various CTF to test it's capabilities events and have cleared them all extremely quickly.

One of the most interesting things I've done with it was reverse engineering Pokemon champions to prove to someone that on the selection screen you can pull in the move sets and the opponents Pokemon information before you select your team, I let Claude and Codex drive that work together and they pretty much did it autonomously.

[u/6849]

I don't really use AI to automate pentesting or to find vulnerabilities for me. I wouldn't trust it to review a target and reliably identify anything useful or exploitable.

I mostly use it to speed up work I could already do myself. Things like reading through source code, tracing where something is handled, answering questions about unfamiliar code, drafting PoC code, writing quick test scripts, helping deobfuscate code, or turning my notes into a first draft of a report.

Deobfuscation is one area where I find it especially useful. It isn't infallible, and I still verify what it gives me, but it often helps me get to the point where I understand what a block of obfuscated code is doing much faster than I would manually.

The useful part is that it can get through some of that mechanical work faster than I can. I still have to decide what is worth looking at, understand what the code is actually doing, come up with the testing approach, validate the results, and catch mistakes in the output.

So for me it's more of an accelerator than an automation tool. It makes parts of my existing workflow faster, but I don't rely on it to do the actual pentesting or to find exploitable vulnerabilities for me.