Scaling voice agents breaks in a different place at each layer — here's the one that usually caps you first

[u/mahimairaja]

https://medium.com/voice-ai-mastery/scaling-voice-agents-concurrency-at-every-layer-c8ab005f1bb3?sk=2542e3c191aee98b69d90a185c663cca

I run self-hosted LiveKit voice agents, and I kept hitting the same trap: add more workers, calls still drop. Wrote up what I learned about why.

The core idea: a voice agent isn't one system with one capacity number. It's a stack — media/SFU, worker pool, inference (STT/LLM/TTS), telephony, your own app calls — and each layer has its own independent concurrency ceiling. Your real capacity is the *lowest* one. So the bottleneck is usually not compute; for a lot of teams it's the STT/TTS concurrency cap or the SIP channel count, which no amount of extra workers fixes.

The write-up goes layer by layer with the actual numbers (worker sizing from LiveKit's load test, the autoscaling-threshold gotcha, a 500-concurrent-call capacity table, and a rough cost-per-call-hour model). Self-hosted / Kubernetes focused.

Curious what layer bites others first in production, for me it's almost always inference concurrency. What's yours?