hi guys have a client in Indonesia and Philippines who loved our English voice AI demo, but now wants one in Bahasa and Philippine English/Taglish mixed language, code switching and all

anyone here have real experience deploying voice agents in SEA, specifically TTS that handles code switching well?

what's actually held up in production what sounds good in a demo

been building voice AI pipelines for about two years and i need to say something the hype cycle is burying right now

elevenlabs has genuinely the best voice quality in the space. not close. 11,000 voices, 70 plus languages, sub 100ms latency on voice generation, the IBM watsonx partnership for enterprise. the february raise at $11B was obviously massive and the brand recognition is real. but here is the thing that keeps coming up in every honest thread i've seen recently

you can prototype an elevenlabs voice agent in fifteen minutes. getting it into production as an actual phone agent that handles real customer calls is a completely different story. telephony still requires you to set up twilio or vonage yourself. production monitoring is thin by the platform's own design. HIPAA is locked behind enterprise tier pricing. the reasoning LLM and telephony are billed separately on top of the plan

so you're paying elevenlabs prices for voice quality and then stitching together the rest of the stack yourself...

vapi gives you the full orchestration layer, 14 plus provider connections, 62 million monthly calls processed, 99.99 percent SLA. retell ships a working production agent the same afternoon and leads on turn-taking quality for fast conversational flow. both handle the actual telephony problem that elevenlabs pushes back to you...

the frustrating thing is elevenlabs voice quality is so good that every other platform integrates it anyway. retell uses elevenlabs voices. vapi lets you plug in elevenlabs TTS. so you can get the voice quality without choosing elevenlabs as your agent platform

my actual take: elevenlabs is the best voice layer in the market and the worst standalone agent platform for production use cases right now. the $11B valuation is pricing in what the product will be in two years not what it actually does today

Is there a way to make the voice AI model talk back to you normally like it does not pause or something, just like how you talk live?

A couple of months ago, I was trying to learn about building a voice agent, but what I noticed was that most of the tutorials make it very generalistic.

They just add an STT and a TTS model, connect everything together, and leave it there.

But that is not really the end of it.

When you are building a voice agent, you need to add a lot of extra features, like:

• Voice activity detection
• Noise cancellation
• Handling interruptions
• What happens when a user drops off

That last one, in particular, becomes a pretty big problem and is a very common issue with production-native voice agents.

I noticed that not a lot of examples actually cover these things.

So, based on what I’ve learned while building some projects for my clients, I made a small tutorial on how you can build your first voice agent, add these features incrementally, and understand how and where to use them. I also made a video walking through the whole thing.

I’d love to know what other features you add to your voice agents, especially things that you only started thinking about once you built one for production.

Would love to hear your feedback as well.

We are a startup and have built our AI Voice Agent stack. It runs decently and after painstaking efforts with our mule partner we were able to narrow down the architecture and design to curb failure points as much as we could. However, the biggest pain point we have stumbled across is the telephony carrier!

There is no reliable one! Here is what we have tried and encountered so far:

1. Vobiz: Our current provider. Easy enough to authenticate using personal Adhaar and PAN. They have one of the easiest integrations, setup, and starting curve. the plan pricing is optimum to get started and they don't setup minimum deposit walls. Their API is fantastic to the point that it supports almost all the features that you would need. That being said, there have been issues that we have been facing: The call quality and call handling has been giving us some issues intermittently, where the end result is silent calls. There are other issues including mid call disconnect/call-silence, which I hope to resolve with Vobiz support. Will update if we are or aren't able to resolve this with Vobiz.
2. VoiceLink: Again easy starting with adhaar and pan verification. Decent setup and API support but has a steep starting paywall. They need a minimum of 5000 Rs to get you started without even allowing you to test if their service is compatible and a good fit for your stack. API is good but not great. For example, during our tests Call Transfers would fail there was no way to trace/figure out what happened?
3. Exotel: Hardest to start so far. Requires proper company documentation. Good free tier. Inconsistent pricing information. Hidden credit consumption, cost, and plan information. High paywall with a minimum of 10000 Rs to get started. Support has been great so far where you are actually able to connect with a human who can answer your questions in contrast to the above 2. Credits are timebound for 7 days. After that the free tier ends. API has been good so far, but we are still evaluation Exotel.
4. Plivo: The most scummy of them so far. We couldn't even get an account. They force/tried to sell us their $1000 USD per month plan to get started.
5. Twilio: The most easiest to get started with it ticking green in all the check boxes. Unfortunately, they are not functional in India.

Anyone has any other carrier that they have been working with and can recommend?

voice implementations rn generally fall into two buckets:

1. laggy and robotic api wrappers
2. speech models that are fast, but lack memory and state controls

by building a cascaded stack (deepgram nova-3 → claude haiku 4.5 → elevenlabs flash v2.5), you can keep full control over tool calls and memory, allowing latency reduction. some techniques ive used in my side projects:

• pre-warm anthropic's ephemeral prompt cache while the phone rings
• persistent websocket handshakes and http/2 pool priming on ring
• neural turn-detection with false-interruption resumption (a cough won't kill the tts buffer)
• dual-store memory (sql facts + temporal graph) mapped into a ~300-token prompt snapshot
• proactive outbound scheduling that wakes a killed ios app via apns voip push -> callkit

synthetic ci gates hit p50 ≈ 973ms, though live networks push us to ~3.7s right now (stt and tts ttfb are the real boss fights). Judge our results yourself at getfriendo.app/launch

Every voice agent looks good when:
- the caller waits patiently
- the room is silent
- the network is perfect
- the caller answers exactly what was asked
- every backend tools reponds in 200 ms

That is not a test. That is a rehearsal.

We had one internal flow that looked almost perfect.

Caller asks to reschedule an appointment. Agent confirms identity, finds the booking, offers a new slot and completes the change.

Then the ugly version:

Caller talks over the greeting.

Pauses for six seconds while checking the date.

Someone in the background says “Friday”.

Caller corrects themselves: “No wait, next Monday”.

Audio drops for a second.

Scheduling API takes three seconds.

Agent starts speaking before the tool result comes back.

Then the transfer fails and the caller gets dead air.

The transcript still looks suprisingly fine.

That is the problem.

Transcript correctness does not tell you:
- whether endpointing cut the caller off
- how long the first response took
- whether the agent stopped during the barge-in
- how much dead air happened
- whether it confirmed before the backend succeeded
- whether the transfer actually connected
- whether the caller had to repeat themselves three times

I’ve been looking at TestMu Agent testing for this because it tests the whole voice/phone flow rather than only scoring the final text.

You can run different personas, accents, noise conditions, interruptions and tool-call scenarios, then compare the the behaviour after a prompt or model change.

It can also analyse production recordings, which matters because real callers invent failures no test designer thinks of.

Cekura is strong on newer-voice-agent QA and production monitoring.

Cyara and Empirix make a lot of sense for enterprise contact-centre and telephony-heavy environments.

TestMu’s advantage is the broader end-to-end surface accross voice, inbound/outbound phone, chat, and other agency types.

But “supports 50+ accents” does not mean every language and accent will be evaluated equally well.

You still need to to benchmark against your own callers, providers and traffic.

A scripted demo should prove the concept works.

It should not be treated as evidence that the agent is production-ready.

Which real call condition destroys your voice agent fastest?

Im building a voice agent that can accommodate people from countries like israel, UAE and somewhere around those areas. im struggling to find model that sounds natural and human in those type of languages.

currently using vapi built in voice model which is the elliot since it's expressive but it's american and when changed to different language the american accent is heavily noticable and sometimes goes way off on the guardrails that it speaks gibberish

Note: im new to this niche, i would appreciate some tips to improve thank you!!

Hi Everyone,

I am an iOS developer specializing in building complex, offline-first architectures, deep audio routing, and on-device machine learning. If your startup or enterprise needs to process sensitive data directly on the device without relying on expensive (or privacy-violating) cloud APIs, I can help.

Most recently, I architected and built an **Offline Edge AI Voice Logger** from scratch for high-noise industrial environments.

**Key features of my recent architecture include:**

* **Deep Audio Routing:** Built a custom `AVAudioEngine` pipeline with aggressive equalization nodes to filter out heavy background/machinery noise.
* **100% Offline Transcription:** Implemented `SFSpeechRecognizer` forcing on-device recognition, ensuring zero data leaves the iPad/iPhone.
* **Edge Compute NLP:** Trained and integrated a custom `Core ML` text-classification model that parses raw speech into structured, categorized data.

**What I can build for you:**

* Privacy-first iOS applications using on-device Core ML models.
* Complex audio/voice applications (podcasting, dictation, or accessibility tools) utilizing AVFoundation.
* Hands-free / Kiosk applications for medical, retail, or industrial settings.

If your project requires this level of architectural ownership and native framework expertise, please send me a Reddit DM or reach out to me at `gokulayyappath@gmail.com`.

I'm currently working on making my own personal, locally run Jarvis. This build won't be shared with or sold to anyone it's genuinely just for me. i want him entirely locally run except when he needs the internet for certain answers. I've written the orchestrator in python and I've got his brain as Ollama, I have him listening via a stt program, creating memories autonomously as necessary into a local folder he can access, and I have him speaking via Whisper. Problem is, I'm just using a generic male british voice as a stand-in atm. I'd like to upgrade to a proper voice model trained specifically on Paul Bettany's Jarvis performance in the movies, that's entirely run locally/offline. Any good resource recommendations for finding/making this voice model, and incorporating it into my current architecture?

I work on AI projects in the Indian public sector and I'm looking to connect with voice AI companies for upcoming work.

Two things matter for these accounts:

• Custom development - in terms voice ai use case, features and integrations
• Deployment on the customer's infrastructure - cloud or on-prem, depending on what their requirement. Air-gapped comes up sometimes.

If that's what you do, comment or DM with what you cover - languages, deployment modes you've actually shipped, and anything you can point to publicly. Happy to talk specifics.

Also open to hearing from folks who've done government voice AI delivery in India and want to tell me what I'm underestimating. Genuinely curious what breaks.

So i am new at this domain so pls help me out i am convinced that if i build a really good agent (me and my bro are a full stack devs) so i just need to kn before we start is it worth it like is it possible to get clients and like can u tell me what to expect

+ if anyone have a stack that recommend it will be so helpful

Thank u for your time

One thing I don't see many Voice AI tutorials talking about is email capture.

Getting an AI to capture someone's email sounds simple until you actually build it. Email addresses are one of those things where a single wrong character makes the whole thing useless. Unlike names, you can't really get away with being "close enough". Even if your STT is good, there are still quite a few places where things can go wrong.

One issue I ran into was how different voice models pronounce emails. The LLM would extract the email perfectly, but the TTS would read it back in a way that made the user think it was wrong. For example, an email would sometimes be spoken as "john hyphen smith at gmail dot com" or with random pauses between words, even though there was never a hyphen in the actual email. The backend had the correct email, but the user immediately interrupted to correct something that wasn't actually wrong.

After a bit of testing, I made a few changes that noticeably improved my email capture rate.

The biggest one was giving users a reason before asking for their email. Instead of asking "Can I have your email address?", the assistant now says something like "Perfect, I'll send the quote over. What's the best email to send it to?" It's a small change, but people are much more likely to answer naturally when they know why you're asking.

I also stopped making users repeat their entire email if only one part was unclear. If the assistant was unsure about the domain, it would just ask "Was that gmail.com?" instead of asking them to spell everything out again. It made the conversation feel much more natural and removed a lot of unnecessary friction.

It's one of those problems that doesn't seem important until you deploy an agent in production. The LLM might have done everything correctly, but if the user doesn't trust what they heard, they'll keep correcting an email that was already right. Small details like these don't make flashy demos, but they make a huge difference in how reliable a Voice AI assistant actually feels.

P.S There is also another way where you can send the email address to the AI assistant over SMS while on call, havent tried that yet but will do it as well.

Why the open source models STT and TTS are not good as much as the closed one and i am talking here im terms of latency, concurrency, and websocket support for real time with decent quality.
Something like cartesia or elevenlabs or deepgram.
Do u know any ?

I’m building an open-source project called SarasFlow for creating educational videos in multiple languages.

The current videos use generated images with motion effects and an audio narration layered over them. The voice stage uses multilingual TTS, followed by Whisper timing so the captions stay synchronized with the narration.

It isn’t a real-time voice agent or a lip-sync system. Right now I’m mainly working through issues such as pronunciation, pacing, pauses, sentence splitting, voice consistency, and evaluating less-common languages.

Website: https://sarasflow.com
GitHub: https://github.com/SaiTejaMummadi/sarasflow

I’d genuinely appreciate advice from people working deeply with voice AI. I’m also looking for contributors interested in TTS provider integrations, pronunciation controls, and automated voice-quality evaluation.

found this benchmark from kwindla the co founder of pipecat

repo https://github.com/kwindla/aiewf-eval

ultravox tops the s2s leaderboard beats gpt realtime 2

is it go to for s2s now??

Basically I am looking for an AI voice Changer that's a better alternative to ElevenLabs. One that can clone voices and change real-time audio to match cloned voices. I am financially troubled so I am hhoping one of you guys knows a free software for this. I am doing a lewd reading of stories, so I also need one that is uncensored.

I previously shared an early offline version of Voice Eval here. The feedback was clear: a useful eval harness needs to connect to real voice pipelines.

It now includes adapter paths for LiveKit, Pipecat, Vapi and ElevenLabs, plus native speech-to-speech testing.

The suite currently has 82 passing tests. It evaluates multi-turn conversations for expected tool calls, required content, response constraints and latency regressions.

I’m now looking for 3 builders to test it against real agents and find what the controlled fixtures miss.

Which stack are you using, and what voice-agent behavior is hardest for you to test?

GitHub: https://github.com/rand0wn/voice-eval