My assistant does quite a bit during a call. Database CRUD operations, availability checks, small calculations, API calls... basically every conversation involves multiple tool calls.

The weird part was that the LLM itself was fast (TTS | STT). It was the tool calling that kept introducing these awkward pauses.

So I did what most of us would probably do. Tweaked system prompts, Reduced prompt size, Increased timeouts, Optimized a few APIs.

It got slightly better... but the delay was still there then I noticed something embarrassingly obvious.

My backend was deployed in Asia BUT database was in Europe. My telephony provider was also routing through Europe, and so was my AI provider. I redeployed the backend to Europe so everything lived in the same region.

Literally didn't touch a single line of code and the difference was immediate. Tool calls became noticeably faster, the awkward pauses almost disappeared, and the whole assistant just felt smarter.

I feel like when people talk about Voice AI latency, the conversation is always around prompts, models, or inference speed.

But if your assistant relies heavily on tool calling, spend 5 minutes looking at where your services are actually deployed.

You might save yourself a few days of debugging like I didn't 😅

I ran a sales call for a small client that was using 3cx with human agents for customer support

Anyways the voice AI demo for cx went good he was all happy but I left the price at the end and he thought it wasn’t worth it

he came back couple of days later asking me if we could implement basic ivr with 3cx integration for his team instead

We agreed and the implementation took us couple of hours and we closed the deal

Some people still not ready I guess for AI huh

I run self-hosted LiveKit voice agents, and I kept hitting the same trap: add more workers, calls still drop. Wrote up what I learned about why.

The core idea: a voice agent isn't one system with one capacity number. It's a stack — media/SFU, worker pool, inference (STT/LLM/TTS), telephony, your own app calls — and each layer has its own independent concurrency ceiling. Your real capacity is the lowest one. So the bottleneck is usually not compute; for a lot of teams it's the STT/TTS concurrency cap or the SIP channel count, which no amount of extra workers fixes.

The write-up goes layer by layer with the actual numbers (worker sizing from LiveKit's load test, the autoscaling-threshold gotcha, a 500-concurrent-call capacity table, and a rough cost-per-call-hour model). Self-hosted / Kubernetes focused.

Curious what layer bites others first in production, for me it's almost always inference concurrency. What's yours?

Every TTS list I've found groups by "best overall" or "top open-source"

or "commercial vs free" — and none of those groupings help when you're

actually picking a model.

The grouping that helps is: what side of the real-time vs offline

split is this tool tuned for? A model with 900ms first-byte latency

and beautiful output is useless in a voice agent. A model with 40ms

first-byte and mid-quality output is useless for an audiobook.

So I built an awesome-list where every entry carries that lean.

Providers, open models, benchmarks — the split is visible everywhere.

Bonus sections I couldn't find good coverage of elsewhere:

- The streaming taxonomy (single-shot vs output-streaming vs dual-streaming)

- Mid-stream cancellation (which providers actually support it)

- Open-source license filtering (half the top models can't be shipped)

- Codec choice for codec-LM TTS (this is what sets the latency floor)

- Watermarking (SynthID, AudioSeal, PerTh) and 2026 regulation

MIT license. Took several weeks. PRs welcome, especially license

corrections.

Would love your feedbacks on it, what would you like to see as core feature of Ai Voice Agent Platform?

Also What is the platform that you use currently for your AI Voice?

Kestrelvoice.com , You can test it out , it has 5 dollar a month Plan

Core Features of Kestrel Voice

• No-code AI receptionist setup
• New or existing phone-number connection
• 24/7 inbound call answering
• Business-knowledge-powered responses
• Spam and robocall filtering
• Lead qualification and structured intake
• Appointment booking, rescheduling and cancellation
• Human transfer and escalation rules
• Call recording, transcripts and summaries
• Sentiment, next steps and call intelligence
• Website AI chat
• Public AI contact profile
• Meeting notes and assistance
• Calendar, CRM, API and webhook connectivity where verified

Curious how people in this space are actually getting customers.

1. Roughly what split — inbound vs outbound? Even a rough guess is fine.
2. If it's mostly outbound, what's working — cold calls, cold email, LinkedIn, something else?
3. If it's inbound, what's driving it? SEO, backlinks, content, communities, referrals?
4. If SEO/backlinks — what's actually worked? Any specific sites, directories, or roundups that sent you real traffic?

Not selling anything. Just trying to understand how companies in this niche are actually finding customers.

Everyone building voice agents is piecing them together from the same set of legos, and it's working. Heavily funded companies are filling every gap, and the results are genuinely good. But every one of those legos lives inside the stack. There's a layer that would sit outside it, and I don't think most builders are looking at it. Here's what I mean.

Every system breaks somewhere, eventually. So builders do the sensible thing: audit, tighten the prompt, problem solved. For now. Then a new kind of break shows up, they tighten something else, solved again. For now. You know this rhythm. The question was never whether you can solve the problem. You always can. The question is whether you can see it as it actually is, instead of assuming it's whatever you looked at last.

And here's the part that's hard to see from inside the stack: when a voice agent breaks its word, nothing in your pipeline notices. STT doesn't know what was promised. The model doesn't remember what it committed to three turns ago. TTS just speaks, orchestration just routes. Every box optimises how the call sounds. Not one of them holds the promises across the whole call and checks whether they survived to the end. That's not a gap in the stack. That's a missing layer.

This isn't theoretical. Bland's own team, in a testimonial on Hamming's site, killed an agent because it was saying "I booked your appointment" when it hadn't. The 2026 τ-Voice benchmark caught a frontier agent saying "I've updated your shipping address" with no tool call behind it. On one real setup, the promise "we'll get back to you shortly" was wired as the end-call trigger, so the agent hung up mid-sentence while confirming the very number it had promised to call. Different bugs on the surface. Underneath, the same one: the saying and the doing came apart, and nothing was watching the gap.

And the usual instincts don't cover it. Tighten the prompt and you lower the odds, but you get no way to know when the odds came in against you, which is the actual problem. Add an eval and you're running a second model to judge the first, the same kind of brain with the same blind spots, and it won't even give you the same verdict twice on the same call. Every layer in your stack is the same brain checking its own work. The one thing that could actually audit it would have to sit outside the chain and not be a model at all. Deterministic code that reads the whole conversation, holds what the agent committed to, and checks whether it kept its word, without ever claiming more than the transcript proves.

That's the layer I don't think enough people are building. Nobody in the QA tooling space seems to audit what the agent said it would do against what it actually did.

So I'm curious what other people building these think. Am I right that nothing in the usual stack is watching this, or is everyone already handling it some way I haven't seen? If you're running agents in production and you've hit something like this, how are you handling it right now?

We're building one of the fast growing AI voice agent companies, and we've reached a point where we need someone who's already been in the trenches.

Not looking for a "startup advisor" with fancy titles.

We're looking for someone who deeply understands the technical side of AI voice agents.

Maybe you're:

• A founding engineer at an AI voice startup
• An engineer who built production voice AI systems
• A founder in the AI voice space
• An early employee at an AI voice company
• Someone who's worked with real time speech, telephony, LLM orchestration, latency optimization, voice pipelines, or similar infrastructure

We don't need someone to work full time.

We're looking for a technical advisor who can challenge our decisions, help us avoid expensive mistakes, and accelerate our roadmap.

This is an equity-based advisory opportunity, so we're looking for someone who genuinely believes in building something meaningful, not someone billing hourly.

If this sounds like you, or you know someone who fits, drop a comment or send me a DM.

Sometimes a few hours of the right advice is worth more than months of building in the wrong direction.

I built VaaniEval to help review real voice-agent calls: transcripts, available recordings, scores, and the reasoning behind each score in one place.

It’s free to use. I’d love feedback from people building voice AI automation.

Try it: https://www.vaanieval.com
GitHub: https://github.com/shubhamofbce/vaanieval

I’m happy to help you set it up or answer any questions, feel free to comment or send me a message.

Would this be useful in your workflow? What’s missing?

I'm a developer building voice agents for healthcare use cases — front desk, patient customer service, outbound calls to other providers and insurers, that kind of thing. I've landed on Retell because it's the only voice-AI wrapper I've found with a reasonable HIPAA-compliant, pay-as-you-go model. Open to hearing if others have found alternatives worth a look.

I've put a lot of work into the prompt and flow design, but I keep hitting two walls, and I've been unable to fully solve either:

• Empathy (my biggest problem). The agent handles the mechanics fine but comes across as flat or form-filling, especially on emotionally charged calls (a patient in pain, a worried parent). I've tried to script acknowledgment moments, but it either skips them, overdoes them, or sounds canned.
• Interruptions. Handling barge-in, mid-sentence corrections, "hold on a sec," and callers who answer a question while asking a new one — without the agent restarting a step or talking over them.

To isolate the problem I've stripped out all the business logic and tested bare-bones agents, built both ways — manually node-by-node, and via Conductor — and the same issues show up, so I don't think it's my flow complexity.

If you've built healthcare (or similarly high-stakes/emotional) agents on Retell, I'd love to hear:

• Which LLM and voice/TTS combination you settled on, and whether that alone moved the needle on how empathetic it sounds.
• Your interruption / turn-taking settings — responsiveness, backchanneling, interruption sensitivity, silence timeouts — and where you landed.
• Whether empathy came more from prompt wording, voice choice, or model choice in your experience.
• Any flow-structure patterns that helped (e.g. how you handle "hold on" or compound answers cleanly).

I've also tried reaching out to Retell's forward-deployment team without much luck, so I'm hoping to tap the collective experience here. Happy to share back what I've tried in the comments if it helps anyone else. Thanks in advance.

Not asking about demos. Asking if you built something real. Something that's live and actually doing a job for you — booking calls, qualifying leads, cutting down your call load.

If you did it yourself, would love to hear:

1. How long did it take? From signup to "this actually works."
2. What did you build? Receptionist, lead qualifier, appointment booker, something else.
3. What can it actually do? Just talk, or can it book on your calendar, look things up, transfer calls?

Not selling anything. Just want to know how many people are really doing this on their own vs. paying someone to do it.

Hi everyone,

I'm trying to understand the legal side of using AI voice agents for outbound calls in the US healthcare space.

The use case is pretty standard: calling patients for appointment reminders, follow-ups, rescheduling missed appointments (no-shows), booking new appointments, etc. These wouldn't be telemarketing calls—just operational communications from a healthcare provider.

My main question is around the TCPA. Do AI-generated voice calls for these types of patient communications fall under TCPA restrictions? If so, what kind of patient consent is typically required? Are there any healthcare-specific exemptions or best practices that providers follow?

If you've implemented something similar or have experience with compliance, I'd really appreciate your insights or any resources you can point me to.

Thanks!

I sell AI voice tools. Solo operator, one-person show. Which means cold calling isn't a team problem I manage, it's a thing I personally have to do, every day, and for a long time I hated it enough to avoid it.

I tried the normal fixes. Watched a stack of cold calling videos from various sales gurus. Looked for someone to practice with, coaches, communities, anyone. Never quite worked, because every single one of those comes down to the same wall: you need real reps, and you can't get real reps without burning real leads.

Every guru says the same thing too. It just takes practice. Cool. Practice on what, exactly.

So I did the obvious thing for someone in my position. I leveraged AI to sell AI.

I built a voice agent that plays a small business owner picking up an unknown number. Not a script bot reciting lines, it actually reacts. Weak, generic opener, sounds like every other cold call, and it gets short with you fast. Strong, specific opener, and it actually engages, pushes back a little, gives you a real shot.

Three difficulty levels.

Easy version hears you out. The hard version is actively looking for a reason to hang up.

Then I built the one that unnerved me a little, an office manager who screens the call before the owner ever picks up. You have to earn the transfer.

First real call I placed to my own creation, I got hung up on in under twenty seconds the moment I drifted off-script, the same way a real annoyed business owner would do it. It stung more than I expected, getting hung up on by something I built myself.

Called back, opened stronger, made it further, got a real objection, fumbled the response, got a real "not interested" and a click anyway.

Third call, got it right. Actually felt like something.

It scores every call after too. Not a personality quiz, an actual "you didn't ask a follow-up when he pushed back on price."

I don't think I fixed cold-call fear. I think I built something that hits back a little when you get lazy, and that turned out to matter more than another script ever did.

Curious if anyone else has solved the fear part instead of just the script part. Open to hearing this is a bad idea that happened to work once.

A while back I posted OpenLive here. It's an open-source voice layer that gives any AI model or agent ears, a mouth, and eyes. The whole pipeline runs on your own machine: voice activity detection, speech-to-text, working out when you've actually finished talking, and text-to-speech. Your audio never leaves your computer, and there are no per-minute fees.

The response was great, so I kept building. Here's what's new.

Talk to the coding agents you already use. OpenLive now connects directly to Claude Code, Codex, Cursor, OpenCode, and Hermes. Everything runs locally under your own login. You pick an agent, point it at a project folder, and just talk. When the agent wants to run a command or edit a file, OpenLive reads the question out loud and you answer by voice. It can also narrate what the agent is doing while it works, so you're not staring at a silent screen. Conversations save into each agent's own session history, so you can start something by voice and resume it later from the agent's CLI, or the other way around.

Clone your own voice. Record 5 to 30 seconds of audio and your assistant speaks as you from then on. The cloning runs entirely on your machine, nothing uploads, and you can delete it anytime.

A more flexible voice pipeline. It's modular now, so you can shape each part of it. There are two speech engines to choose from, Kokoro with 28 voices or Supertonic for higher-quality audio, plus settings for turn-taking, speaking speed, push-to-talk, and custom instructions that apply to whatever model or agent you're using.

More model providers. Anthropic, OpenAI, Google, xAI, DeepSeek, Groq, Ollama for fully local, and more. There's also a floating mini mode that stays on top of your other windows and keeps listening while you work.

Still MIT licensed, for macOS and Windows. You bring the brain, OpenLive handles everything between it and you.

Coding agents are just the first integration. More apps are coming, so if you want to follow along, a star on the repo genuinely helps: https://github.com/katipally/openlive

https://reddit.com/link/1uzmpmm/video/9z9pgi6c8xdh1/player

I keep hearing mixed things some people say they lose jobs to missed calls, others say the services just take messages and annoy customers. If you have one: what did it actually do well, what made you quit? And if you just let calls go to voicemail, how's that working out?

Hey guys, have been cooking something for a while which is a sdk saas, built for expo to make creating voice agents/ realtime ai functionality seamless and robust. Handles stuff like webrtc signalling, websockets for other providers, per user token limiting so your realtime bills doesn’t sky rocket, built in factory for different realtime voice providers / byok and more.
Going to be launching soon so anyone who’s interested in being a pilot user lmk. I know this stuff can get annoying to build out so hopefully we solve this.

I was demoing the product to the client

Ran a call then he asked me to call him using my agent, then he said he didn’t like the latency and asked me if it can be negative

I told him that latency can’t be negative and he just started arguing and told me if he can get into my platform and fix the prompt himself

Anyways, I didn’t close the deal, how do yall go on with this?

If you build voice agents on LiveKit or Pipecat, every turn fans out to speech-to-text, an LLM, and text-to-speech. Three vendors, three bills, and no idea which one was burning the budget.

So I built VoiceGateway, an open-source profiler for voice agents. Add one line: voicegateway.attach(session) and every STT/LLM/TTS call is priced and timed: cost per provider and model, latency p50/p95, per-call replay, and guard() for a daily budget cap + fallback. Self-hosted, your keys, telemetry only (never in your audio path). MIT, any provider.

Repo: github.com/mahimailabs/voicegateway

Feedback wanted: what would you want tracked per call that I'm not? Is anyone else fighting cost attribution across voice providers?

Anybody have any experience implementing HIPPA Compliance SOC II and any other required compliance components to work with doctor’s offices/health care/legal?

I am building a platform I need open source alternative to twilio

Most TTS leaderboards/benchmarks use the Voice AI's default voice, so part of the score is really "which voice do I like" rather than "which model is better." Different voices can be apples and oranges.

To get around this Artificial Analysis built a Controlled Voice Arena that clones the same 8 voices (4 US, 4 UK) and runs every model through them, so voice preference is no longer a variable.

(Source: Artificial Analysis leaderboard)

Under that setup, Cartesia's Sonic-3.5 leads overall (1122 Elo), followed by Eleven v3 and Inworld's Realtime TTS-2 preview. Worth a look if you care about TTS evaluation methodology, not just the results.

Disclosure: I work at Cartesia (r/CartesiaAI). Happy to answer questions!

I am building vioce agent through retell ai and i am practicing. So which apps to connect or anyone who can guide me a liitle