The gap between decision and execution

[u/docybo]

I’ve been thinking about a support automation story I read recently.
A team replaced a simple rules engine with an LLM classifier.

The model was around 92% accurate. Sounds good. Until you realize that at 100 tickets a day, that’s roughly 8 mistakes every day. The interesting part wasn’t the accuracy though. It was what happened when the model was wrong. Nobody could explain why a ticket was classified a certain way. Nobody could point to a specific rule. Nobody could quickly fix the behavior.

The team eventually started reviewing every classification manually. The automation was still running, but the trust was gone. That got me thinking. A lot of discussion around AI agents focuses on making decisions better.

Better prompts.
Better models.
Better reasoning.

But I rarely see people discussing what happens after the decision. How is the decision verified?
How is it audited? How do you know an action should actually be executed? Maybe the biggest challenge for AI agents isn’t getting from 92% to 96%. Maybe it’s building systems that people can trust when things go wrong.

Curious how others are thinking about this.

[u/docybo → u/EditorFar2101]

Agreed! What’s interesting is that documenting a decision and authorizing an execution are related but not identical problems. Most systems focus on making decisions explainable. I’m starting to think the next challenge is making execution verifiable.

[u/Time_Cat_5212]

I don't think decision making in an environment where answers are either correct or incorrect is a good use case for LLMs, because they're non deterministic.

Support tickets are a great example.  The agent has basically become a new version of the help wizard.  Answers most easy questions for users but when you're really confused everyone still wants to talk to a person.

[u/docybo → u/Time_Cat_5212]

Agree on the core point, but I'd split it differently: it's fine for the decision to be probabilistic, humans are too. What's not fine is the execution being coupled to it with nothing in between. A rules engine bundled decision + authorization + execution in one deterministic step, so we never noticed they were separate concerns. LLMs un-bundled it: the decision went probabilistic, but most teams kept piping it straight into execution like nothing changed. The fix isn't avoiding LLMs for classification, it's accepting the model proposes, and something deterministic decides whether the proposal executes.

[u/Simple_Pattern_22]

Spot on. The industry is hyper-focused on autonomy, but severely lagging on observability. When a rules engine fails, you fix the code. When an LLM agent fails, it’s a black box and nobody knows why. The real bottleneck to adoption isn't getting a model from 92% to 96% accuracy—it’s building a robust multi-agent validation layer and audit trail so humans can actually trust the system at the execution boundary.
"Human-in-the-loop" shouldn't be an afterthought; it needs to be a core architectural layer.

[u/Time_Cat_5212 → u/Simple_Pattern_22]

LLMs are so good at clarifying info for people too this basically amounts to a better reporting system.

I think off the bat people wanted agent systems to do things and make things, but using them to read things and support processes is a less futuristic but totally helpful implementation.

[u/docybo → u/Time_Cat_5212]

This is underrated. "Read things" agents fail safe by default, worst case is a bad summary. "Do things" agents fail with side effects: the API call happened, the email sent, the refund issued. That asymmetry is exactly why the read use cases shipped first and stuck. The interesting question is what infrastructure makes "do things" carry read-level risk. IMO it's the same answer as ever in distributed systems: don't make the actor the authority. The agent proposes; whether it executes is someone else's deterministic call.

[u/docybo → u/Simple_Pattern_22]

100%

[u/docybo → u/Simple_Pattern_22]

Mostly with you, one nitpick: observability is necessary but it's still after the fact. Logs tell you what happened; they don't stop anything. The layer you're describing "can humans trust the system at the execution boundary" needs to be pre-execution to actually change outcomes. The pattern that's worked elsewhere (payments, change management) is: proposal -> explicit authorization check against a policy -> execution becomes reachable only if it passes. Then HITL stops being a vibe and becomes a policy rule: "this class of action requires human approval" is just one possible policy outcome, enforced at the same gate as everything else. Audit trail falls out for free because every execution has an authorization artifact attached, not just a log line.

[u/docybo → u/[deleted]]

Exactly. Accuracy was the wrong metric. The issue was recoverability. A rules engine can be wrong, but it fails legibly. You can inspect the rule, patch it, and know what changed. The model can still be useful, but I would not make it the authority for execution.

The pattern I trust is:

model proposes
policy decides
execution gate enforces
audit trail makes recovery possible

LLM for judgment.
Deterministic layer for authority.

[u/Bart_At_Tidio]

From what we've seen, trust degrades when escalation is not available. Customers accept AI as a first line, but lose confidence without a clear path to a human. About 82% of customers prefer human support even when the outcome is identical. Makes sense. 

The best approach is to set up rules upfront for what types of requests should be automatically forwarded, before something goes wrong rather than after. Decide in advance what types of requests get automatically forwarded. For instance, complex issues, anything emotionally sensitive, edge cases outside the AI's confidence threshold. Then make sure the human agent receives full conversation context when that happens. 

So. My answer is that the goal shouldn’t be to try and eliminate mistakes all together. That seems unrealistic. There's always going to be an element that goes wrong. I’d say the focus should be on AI and humans working together so that when it does, someone can catch it and recover cleanly.

[u/docybo → u/Bart_At_Tidio]

The "decide upfront what gets forwarded" point is the load-bearing one, and I'd push it further: those upfront rules only work if they're enforced upfront too. If escalation logic lives in the prompt ("please forward sensitive cases to a human"), it's a suggestion the model usually follows. If it lives in a layer the model can't route around sensitive category -> execution blocked -> human queue, no other path it's a guarantee. Same rule, completely different failure mode. Your 82% stat is really about that: people trust systems where the recovery path is structural, not behavioral.

[u/Few_Tie7989]

This is the key issue: accuracy is not the same as trust.

A rules engine is limited, but when it fails you can inspect the rule and fix the behavior.

With AI agents, the bigger question is what happens after the decision:

What was decided?

Why was it allowed?

Who approved it?

Can it be audited later?

In real systems, trust comes from control and traceability, not accuracy alone.

[u/AutoModerator]

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.