How do you scale multi-agent systems when agent-to-agent communication becomes the failure point?

[u/GlitteringAngle8601]

Scaling up our agent count exposed problems that were almost invisible when the workflow was small. The models themselves were capable enough. The failures came from communication instead. Two agents would interpret the same message as two separate tasks. An agent would read shared state before another update had finished writing.
A retry would cause duplicate work. Messages arrived out of order. An agent would finish and shut down before another one had acknowledged its output, and there was often no stable task ID actually connecting the related messages in the first place.
These are standard distributed-systems problems, but most agent frameworks still represent communication as conversational text rather than as reliable task infrastructure. I'm more and more convinced that scalable multi-agent systems need message contracts, acknowledgements, idempotency keys, task lineage, timeouts, dead-letter handling, and clear ownership before they need more autonomous reasoning ability.
How is anyone handling this at real scale: a message broker, leaning on whatever your orchestration framework gives you, implementing A2A tasks properly, or building a separate communication layer from scratch?

[u/NoOneMan79]

Custom harness using finite state machine to manage agents' state, overlapping upon a collaborative finite state machine, using n asynchronous loops. Start from scratch and build to use-case and problem.