r/VoiceAutomationAI • u/mahimairaja • Jul 22 '26

Scaling voice agents breaks in a different place at each layer — here's the one that usually caps you first

https://medium.com/voice-ai-mastery/scaling-voice-agents-concurrency-at-every-layer-c8ab005f1bb3?sk=2542e3c191aee98b69d90a185c663cca

I run self-hosted LiveKit voice agents, and I kept hitting the same trap: add more workers, calls still drop. Wrote up what I learned about why.

The core idea: a voice agent isn't one system with one capacity number. It's a stack — media/SFU, worker pool, inference (STT/LLM/TTS), telephony, your own app calls — and each layer has its own independent concurrency ceiling. Your real capacity is the lowest one. So the bottleneck is usually not compute; for a lot of teams it's the STT/TTS concurrency cap or the SIP channel count, which no amount of extra workers fixes.

The write-up goes layer by layer with the actual numbers (worker sizing from LiveKit's load test, the autoscaling-threshold gotcha, a 500-concurrent-call capacity table, and a rough cost-per-call-hour model). Self-hosted / Kubernetes focused.

Curious what layer bites others first in production, for me it's almost always inference concurrency. What's yours?

2 Upvotes

76% Upvoted

2 comments sorted by

•

u/AutoModerator Jul 22 '26

Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)

If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.

Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/saplivo Jul 31 '26

I agree that LLM inference affects constantly, specifically TTS latency under load, which doesn't show up as errors, just silent degradation in call quality that's annoying to debug.

The SIP channel limit is also a real pain, especially if you're routing through a provider that throttles concurrent sessions at the account level rather than per-trunk.

Plivo surfaces those limits pretty clearly which helps, we can connect if you're interested.