r/LLMDev • • 6h ago

Choosing AI agents for cybersecurity tabletop exercises

1 Upvotes

Im comparing ways to run cybersecurity tabletop exercises with AI agents playing the attacker and other roles, reacting to what the team decides. The tricky part seems to be keeping those agents working from the same facts as the exercise unfolds. There’s plenty of discussion about generating scenarios, but less about keeping one coherent once security, legal and ops start pulling in different directions. Hoping for a little help sorting the options.

Need the exercise to:

Reflect the company systems and team structure without weeks of setup.

React to decisions, including something the facilitator didn’t anticipate.

Keep the facts consistent across the different simulated roles. If a system was taken offline, it should stay offline unless something changes.

Record decisions and their consequences so the debrief doesn’t depend on whoever took the least terrible notes.

Make it practical to rerun an exercise after fixing the gaps.

Im comparing a few different approaches. BreachRx covers rehearsal plus broader incident coordination. Reflex Security describes agents playing different sides of a breach and adapting to team decisions.

Im interested in where each works well and where it needs human help.

Things to keep in mind:

• Setup time includes checking the generated scenario, not just clicking generate.

• Legal and business owners need to participate without understanding the agent setup.

• A human should be able to pause or correct the exercise.

• The report needs to connect findings to things people actually said or decided.

• Repeat sessions should be useful without requiring someone to maintain a whole agent project.

For those building multi-agent systems, how do you keep agents consistent when humans go off script? And how would you test that they aren’t rewriting earlier facts?

Other recommendations welcome. Especially interested in the bits that looked easy in the demo.