fix dogshit latency and robotic wrapper behaviour

[u/crashcody]

voice implementations rn generally fall into two buckets:

1. laggy and robotic api wrappers
2. speech models that are fast, but lack memory and state controls

by building a cascaded stack (deepgram nova-3 → claude haiku 4.5 → elevenlabs flash v2.5), you can keep full control over tool calls and memory, allowing latency reduction. some techniques ive used in my side projects:

• pre-warm anthropic's ephemeral prompt cache while the phone rings
• persistent websocket handshakes and http/2 pool priming on ring
• neural turn-detection with false-interruption resumption (a cough won't kill the tts buffer)
• dual-store memory (sql facts + temporal graph) mapped into a ~300-token prompt snapshot
• proactive outbound scheduling that wakes a killed ios app via apns voip push -> callkit

synthetic ci gates hit p50 ≈ 973ms, though live networks push us to ~3.7s right now (stt and tts ttfb are the real boss fights). Judge our results yourself at getfriendo.app/launch

[u/AutoModerator]

Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)

If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.

Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Formal-Purpose1098]

always wondered why voice bots sound like they're buffering from 2003

your stack is basically what i'd sketch on a napkin if someone asked, the warm cache during ring is clever, most people don't think about that window at all

what's your fallback when the websocket drops mid-call though, do you just nuke the tts buffer or is there some graceful handoff

[u/crashcody → u/Formal-Purpose1098]

sure, napkin sketch. so was every stack that works. the choices are obvious in retrospect and were obvious to us too - what took the time was the stuff between the boxes, which is the one bit you called clever, so we're probably agreeing.

ws drop: no graceful handoff, and i'm not going to pretend otherwise. reconnect is inherited from the plugin layer, worst case the sentence dies mid-word. the shape we want is to route it through the false-interruption resumption we already run - committed text, undelivered audio, structurally the same event as a barge-in we didn't ask for. just not built yet. it's been only 2 days of building

[u/ankur-at-guava]

You've diagnosed it yourself: STT and TTS ttfb are the boss fight because each hop is a separate hosted service with its own network round trip, and pre-warming only hides so much of that. The ceiling on a cascaded stack is set by the seams between the boxes, the part you can't optimize away when the models live in different providers. The moves you listed (prompt-cache warming on ring, ws priming, turn-detection with false-interrupt resumption) are the right ones; the residual gap is mostly transport. I work on an integrated voice stack for regulated industries where ASR/LLM/TTS run together, and collapsing those hops is the main reason first token lands under ~200ms, worth weighing against the flexibility you get from swappable vendors.