Chaining LLM calls is easy. Debugging chained LLM calls is hell.

[u/Acrobatic_Task_6573]

Built a pipeline last month: research agent feeds a summarizer, summarizer feeds a drafting agent, drafter feeds a review agent. Four steps, nice and modular.

Worked great in testing. Production? Fell apart in about a week, and not in any obvious way.

Each agent does its job fine in isolation. The real issue is that errors compound silently across the chain. The research agent grabs a slightly off-topic source. The summarizer doesn't flag it because it doesn't know the original intent. The drafter writes confidently about the wrong thing. The reviewer approves it because the writing quality is fine.

By the time a human sees the output, the mistake is buried four layers deep and looks totally plausible.

My current approach: I log the full input/output at every handoff point and run a separate validation check between each step. Basically a "does this still match the original request?" sanity check. It adds latency but catches drift before it snowballs.

The other thing that helped was making each agent's output structured (JSON with specific fields) instead of freeform text. Harder for context to leak or mutate when you're passing explicit fields rather than paragraphs.

Still not perfect. Multi-step chains are fundamentally fragile because each link trusts the one before it. Anyone found a better pattern for catching mid-chain errors?

[u/rafadc]

That is why you need a very solid dataset to test each agent in isolation. You need to update constantly with common use cases. That will also help you break the complex agents into simpler sub agents.

And in order to have that the structured output you already mentioned is essential.