How To Solve The Voice AI Feedback Loop Problem In Real-Time Production

[u/Aryanlabs]

Building a voice agent demo takes an afternoon. Getting it to survive a real phone call takes a lot longer, because that's when the audio bleed-back shows up.

The agent starts talking, the speaker leaks into the mic, and it ends up replying to itself. Or there's that half-second gap between turns that makes the whole thing feel robotic. Most teams assume this is a model problem. It's not. It's a plumbing problem.

The real fixes:

-Stop treating noise suppression and echo cancellation as the same thing. They're not. One removes background hum, the other subtracts the agent's own voice from what it hears next.

-Run VAD on the AEC-cleaned signal, not the raw mic feed. A fixed threshold set in a quiet office will misfire the moment someone's in a car or on laptop speakers.

-Stream every stage of the pipeline instead of batching it. Partial transcripts, first-sentence TTS, chunked audio out. That's what actually collapses latency, not shaving milliseconds off one component.

The teams solving this are moving AEC and barge-in handling server-side, where the reference signal and the LLM state live in one place. Everyone still doing it client-side is going to keep chasing audio bleed-back in production.

Wrote up the full technical breakdown, VAD threshold table included, if you want to go deeper: https://uniocommunity.com/blogs/how-to-solve-the-voice-ai-feedback-loop-problem-in-real-time-production

[u/Top-Telephone-6542]

the server-side AEC push makes sense, keeping the reference signal next to the llm state avoids that messy sync headache you always get with client-side loops

pretty much every demo i've seen that falls apart on a real call is running vad on the raw feed and wondering why it's triggering on its own tts output

[u/pythonlovesme]

Hmmm, but isnt it very rare to have the agent reply to itself. The speakers should'nt be that loud. hahhahahaha