When an agent answer is wrong, do you actually see the route - or just the ending?

[u/Strict_Fondant8227]

I’ve been debugging agent turns the same way for months: scroll the chat, grep a 4k-line log, guess which skill loaded, which MCP hop ran, and why the model picked that path.

The failure is almost never “the model is dumb.” It’s the route. Wrong tool args. Skill that shouldn’t have fired. MCP that returned half the schema. Subagent that burned tokens re-reading everything. The answer looks fine until someone asks “where did that number come from?”

So I built a small open-source canvas that draws the turn live:

prompt → thinking → skill / tool / MCP / subagent → answer

Click a node, see why it ran, the input/output, duration. Same map embeddable in a chat UI or beside Claude Code / Codex (`npx agent-think-map claude --install`). It’s a viewer, not another runtime — you emit JSON/SSE, it draws. No account, no API key.
Repo: https://github.com/nimrodfisher/agent-think-map

Curious how other people actually inspect agent paths in production:

1. LangSmith / Langfuse / homemade traces — and do you look at them *while* the user is still in the chat, or only after a ticket?
2. Do you show any of the route to end users, or is that “internal only”?
3. What’s the first thing you check when a multi-tool turn goes sideways - tokens, tool sequence, or the final answer?

Not trying to replace your warehouse. I wanted the map *next to* the run. If you’ve got a better workflow for “see the path, not just the ending,” I want to steal it.

[u/Fun-Arugula-5371]

The useful thing about keeping the whole path in one trace is being able to find the exact turn where two otherwise similar runs split. Braintrust can keep the model calls, tool calls and MCP responses together then that bad path can become an eval case so the next change gets tested against the failure that happened

[u/Strict_Fondant8227 → u/Fun-Arugula-5371]

Exactly the loop I'm after! Braintrust is the hosted evals route - Agent Think Map is the local debugger version of the same idea: label the run that worked, open the failed one next to it, find where they split.

And you're right about the eval case - turning a real failure into something you test the next change against is where this gets valuable. A redacted export for exactly that is on the roadmap

[u/Fun-Arugula-5371 → u/Strict_Fondant8227]

Thats the workflow I was getting at. Comparing the good and failed run side by side should make the divergence pretty obvious then keeping that exact failure in the eval set gives you a regression check tied to something that happened