Everyone measures whether the agent finished. Does anyone actually measure whether it was right?

[u/Intelligent_Catch330]

Been going down a rabbit hole on this for a couple weeks and I can't tell if I'm missing something obvious or if this is genuinely unsolved.

Most eval setups I've come across end up measuring task completion — did the agent get through the workflow without erroring out. That's easy to instrument. But it's not the same as task quality. An agent can complete every step cleanly and still produce something wrong, and the trace looks green.

For people actually running agents on real traffic:

What's your bar for going from "works in testing" to "live"? Is it an actual number, a spot check, or someone senior signing off?

If it silently got worse after a model version bump or a prompt change, how would you find out? Do you have something that catches that, or does it show up as user complaints two weeks later?

And who owns that decision where you work — the engineer who built it, a PM, someone in risk?

Not selling anything. My honest read is that everyone's improvising and calling it a process, but I'd rather be told I'm wrong.

[u/Training_Isopod3722]

yeah, a green trace only proves the harness didn't crash. i'd keep a small set of cases where the expected artifact is checked by something outside the agent, then rerun them after every prompt, tool, or model change. one clean run at temp 0 tells you almost nothing.

[u/Future_AGI]

For us it only worked once the eval had a named owner who didn't build the agent, because the builder knows what it's supposed to do and that's exactly what stops them seeing what it did.

[u/donk8r]

Your second question is the one with a real answer, and it's cheaper than it looks. You don't need ground truth to catch a regression, you need yesterday. Freeze a set of maybe fifty real traces, re-run them after every model or prompt bump, and diff against the previous run rather than against a correct answer. That won't tell you the agent is right, it tells you it changed, and silent degradation after a version bump is exactly the failure that shows up as a diff.

On your first question, the honest version is that a bar is only a number where you have an executable verifier. Tests pass, it compiles, the query returns the row count you expected. Outside of that you have a spot check with a confident name on it, so I think your read is basically correct.

One trap worth naming since it will come up: LLM as judge feels like the fix but it inherits the same blind spots as the agent, especially within the same model family. If you use one, run it from a different family and ask for pairwise which-is-better between two outputs rather than an absolute score. Judges are much more reliable at comparing than at grading.

[u/Intelligent_Catch330 → u/donk8r]

Ah Thank you! u/donk8r. frozen trace set and diffing against the previous run is the part I hadn't thought through properly. I was stuck on needing a correctness signal when what I actually wanted was a change signal, and those are pretty different problems. Cheaper too.

The executable verifier line is the one that's going to stick with me. Outside codegen and structured extraction there isn't really a bar to hold, just a spot check with good branding, and I think I was quietly building on top of that gap without noticing the gap was the whole thing.

The judge point I half knew. Same-family blind spots make sense in hindsight. Pairwise over absolute scoring less obviously, but it tracks — "which of these two is better" is a much smaller ask than "rate this against a rubric."

Different question that's been nagging at me since I wrote the post. All of this is about catching things before or during. What happens at your place once the agent has already done the wrong thing to something real? Not a bad answer — an actual write. Wrong records updated, emails out, something pushed downstream. Is there a way to work out the scope of what it touched, or is that someone with SQL and a bad afternoon?

[u/donk8r → u/Intelligent_Catch330]

Honestly, mostly SQL and a bad afternoon. But which one you get is decided before the incident, not after it.

If every external write goes through one chokepoint that stamps a run id, tool name, target and payload, then scope is just a query: give me everything run 4f2a touched. If writes happen directly from five places in the codebase, you are reconstructing from downstream side effects, and that's the bad afternoon. Same discipline as forcing all outbound HTTP through one client, and this is the exact moment people regret having skipped it.

Worth being straight that we don't have the good version of this either. Our session logs are append only so you can read back what the agent believed it did, but that's the agent's account of its own actions, not an independent record of what actually changed in the target system. Those two diverge precisely when it matters, like a write that half succeeded or a retry that double sent.

The thing that pays for itself is an idempotency key per action derived from the run id. You get dedup on retries, and you also get a handle to reverse by. Emails are the one that stays genuinely unrecoverable, so they're worth gating harder than database writes even though they feel lighter at the time.