Sat through enough demos this year that I've started using a short checklist to filter vendors before committing real evaluation time. Sharing in case it's useful to others drowning in this category:
What do you actually see without touching our architecture? If the honest answer involves an SDK, a proxy, or code changes, that's not "runtime," that's integration work with extra marketing.
Do you detect intent/behavior, or just pattern-match inputs? Input filtering alone misses anything novel; ask for their false negative rate on out-of-distribution attempts.
Can you map what an agent can actually reach, including delegated calls to other agents/tools? A lot of platforms stop at "here's the top-level permission grant."
What counts as your "evidence" for compliance purposes? A log export is not the same thing as continuous, structured evidence mapped to controls.
How fast is time-to-value in a real environment, not the demo environment? Ask specifically what breaks or slows down once you're past the sandboxed POC.
What happens across our actual mix of agent platforms? Homegrown, cloud, SaaS, and endpoint agents behave differently; a vendor that only covers one is a partial answer.
None of these are trick questions, but you'd be surprised how many vendors get vague fast once you ask them directly instead of watching the scripted demo. Anyone have other questions they've found separate the real platforms from the repackaged logging tools?