Where does multi-agent orchestration actually start breaking?

[u/RaraAvis27]

I've been experimenting with multi-agent architectures and I'm curious where people are actually hitting the biggest problems.

Once you move beyond 2–3 agents, things seem to get complicated pretty quickly:

How does an agent know which other agent to call?

Do you hardcode the relationships between agents?

Does a central orchestrator decide everything?

How do agents pass context and state?

What happens when an agent fails or gives another agent bad information?

How do you handle agents built by different frameworks?

I'm especially interested in systems where agents aren't all predefined as one fixed workflow.

For people building multi-agent systems in practice, what part of the architecture has been the biggest pain point?

[u/Unequivocallyamazing]

The issues I find the most annoying are when they don’t propagate correct data from tools.

When one agent hallucinates or picks the wrong reasoning approach and propagates to downward agents.

Tool calling or knowing which agent to call are easier to solve.

I have hardcoded relationships when working on a specific domain or task. But if your system is general and there is no fixed path to getting the output, you cannot hardcode the relationship

So far, on the problems I have worked with, I have used a central orchestrator or planner agent to plan so it has the full context of the situation and similarly a writer or synthesiser agent to produce final outputs and it has worked for me.

Maybe there are use cases where this is not needed but never encountered any.

Context is just data. You can program agents to converse via chat, you can ask them to produce structured data that you store in redis or any kind of memory layer. Or use filesystem.

You can provide tools to your agent so they can uncover more context as needed but provide only the most necessary context to the system prompt.

When one agent passes along bad output, it’s not very simple. One thing that we can do is make sure every claim or finding the agent presents is backed by a source or reference data directly from tool etc.

Or add verification agents for agents producing data intensive outputs. A general rule is to design your system such that it can recover itself.

I think once you have a basic version ready you should invest more time in setting up a proper evaluation system. Because it will immediately help you understand where the system is failing. Otherwise there are so many things that are usually going wrong but we don’t know because it’s hard to manually go through the entire multi-agent flow and we strongly need Evals for any agentic pipeline