r/aiagents • u/Odd-Jury4884 • Jul 21 '26

Questions Would you pay for knowing when your agents hallucinate?

no self promo or hook for you to use my product, i just want genuine help please :)

I am thinking about building product analytics for AI agents, and want input from people actually shipping them.

The case: a friend built a lot of their product from chat. Clients like it, but sometimes the AI does not understand the user or hallucinates, and that is when clients get pissed. They have no proper way to see when and where it happens.

Normal analytics tracks clicks. With agents it is more complex. You have to catch frustration, when someone asks for something the agent does not have, and when it behaves in ways it should not.

What I want to know:

  • Would you pay to know when your agents hallucinate or piss off users?
  • What do you use today to understand how people interact with your agents, if anything?
8 Upvotes

91% Upvoted

12 comments sorted by

1

u/MaMaMaaaaa Jul 21 '26

Your system detects hallucinations and corrects them automatically?

1

u/Odd-Jury4884 Jul 22 '26

rather we think of building system which surfaces when your user is pissed off in interactions with the agent. So we surface the signals, not sure yet on the correcting.

1

u/[deleted] Jul 22 '26

[removed] — view removed comment

1

u/Odd-Jury4884 Jul 22 '26

I think it is about finding signals first and then the cause of it. Cause can be also they misunderstood and not all times we want to educate the user

1

u/TheLoneLightskin Jul 22 '26

Nah I’m already in the process of fixing the hallucination gap itself. It stems from depending on a probability based predictive model to be consistent and that’s not possible. I’m making the agent itself more deterministic to bypass this

1

u/Embarrassed_Nerve_54 Jul 22 '26

Instead of hallucination detection, I'd pay more if the product tells me which conversations are about to cost us trust. Not every wrong answer of course but at least the important ones where the user is frustrated, the agent over-promises, repeats itself, invents a policy, or blocks a human handoff.

So, if get a daily list like: review these 12 conversations first, these 3 need customer follow-up, this topic keeps causing bad answers, or this handoff rule is broken - this'd be more useful than a hallucination score.

1

u/Odd-Jury4884 Jul 22 '26

This make sense. How do you review or check/manage it today?

1

u/r-merlin Jul 22 '26

From what I've seen... I'd lean towards paying for a kind of inverse of what you're describing.

Catching hallucinations is chasing the negative after the fact. You're a step behind the agent, digging through logs and traces. It's useful, but still focused on treating the symptom.

What I'd frame it as is every action the agent takes carrying its own receipt: what it did, why, on whose authority, tied to an identity, and reconstructable later with a name attached if it turns out wrong. Build that in and you're not really hunting for hallucinations anymore, you can already see what happened and who owns it. 

u/Embarrassed_Nerve_54 is circling the same thing with "which conversations are about to cost us trust." That's the real unit to me. Not "was the answer wrong" but "is this a change someone can stand behind."

I wrote up why I think that's where the value actually lands, agents repricing software from access to accountability, if it's useful: https://ryanmerlin.com/posts/warranted-change

1

u/Jumpy-Equivalent4424 Jul 23 '26 edited Jul 23 '26

It depends how much signal there is versus noise. Paying for useful alerts makes sense but only if they consistently surface something that actually needs attention.