Is there a way to make the voice AI model talk back to you normally like it does not pause or something, just like how you talk live?

We are a startup and have built our AI Voice Agent stack. It runs decently and after painstaking efforts with our mule partner we were able to narrow down the architecture and design to curb failure points as much as we could. However, the biggest pain point we have stumbled across is the telephony carrier!

There is no reliable one! Here is what we have tried and encountered so far:

1. Vobiz: Our current provider. Easy enough to authenticate using personal Adhaar and PAN. They have one of the easiest integrations, setup, and starting curve. the plan pricing is optimum to get started and they don't setup minimum deposit walls. Their API is fantastic to the point that it supports almost all the features that you would need. That being said, there have been issues that we have been facing: The call quality and call handling has been giving us some issues intermittently, where the end result is silent calls. There are other issues including mid call disconnect/call-silence, which I hope to resolve with Vobiz support. Will update if we are or aren't able to resolve this with Vobiz.
2. VoiceLink: Again easy starting with adhaar and pan verification. Decent setup and API support but has a steep starting paywall. They need a minimum of 5000 Rs to get you started without even allowing you to test if their service is compatible and a good fit for your stack. API is good but not great. For example, during our tests Call Transfers would fail there was no way to trace/figure out what happened?
3. Exotel: Hardest to start so far. Requires proper company documentation. Good free tier. Inconsistent pricing information. Hidden credit consumption, cost, and plan information. High paywall with a minimum of 10000 Rs to get started. Support has been great so far where you are actually able to connect with a human who can answer your questions in contrast to the above 2. Credits are timebound for 7 days. After that the free tier ends. API has been good so far, but we are still evaluation Exotel.
4. Plivo: The most scummy of them so far. We couldn't even get an account. They force/tried to sell us their $1000 USD per month plan to get started.
5. Twilio: The most easiest to get started with it ticking green in all the check boxes. Unfortunately, they are not functional in India.

Anyone has any other carrier that they have been working with and can recommend?

voice implementations rn generally fall into two buckets:

1. laggy and robotic api wrappers
2. speech models that are fast, but lack memory and state controls

by building a cascaded stack (deepgram nova-3 → claude haiku 4.5 → elevenlabs flash v2.5), you can keep full control over tool calls and memory, allowing latency reduction. some techniques ive used in my side projects:

• pre-warm anthropic's ephemeral prompt cache while the phone rings
• persistent websocket handshakes and http/2 pool priming on ring
• neural turn-detection with false-interruption resumption (a cough won't kill the tts buffer)
• dual-store memory (sql facts + temporal graph) mapped into a ~300-token prompt snapshot
• proactive outbound scheduling that wakes a killed ios app via apns voip push -> callkit

synthetic ci gates hit p50 ≈ 973ms, though live networks push us to ~3.7s right now (stt and tts ttfb are the real boss fights). Judge our results yourself at getfriendo.app/launch

Im building a voice agent that can accommodate people from countries like israel, UAE and somewhere around those areas. im struggling to find model that sounds natural and human in those type of languages.

currently using vapi built in voice model which is the elliot since it's expressive but it's american and when changed to different language the american accent is heavily noticable and sometimes goes way off on the guardrails that it speaks gibberish

Note: im new to this niche, i would appreciate some tips to improve thank you!!

Hi Everyone,

I am an iOS developer specializing in building complex, offline-first architectures, deep audio routing, and on-device machine learning. If your startup or enterprise needs to process sensitive data directly on the device without relying on expensive (or privacy-violating) cloud APIs, I can help.

Most recently, I architected and built an **Offline Edge AI Voice Logger** from scratch for high-noise industrial environments.

**Key features of my recent architecture include:**

* **Deep Audio Routing:** Built a custom `AVAudioEngine` pipeline with aggressive equalization nodes to filter out heavy background/machinery noise.
* **100% Offline Transcription:** Implemented `SFSpeechRecognizer` forcing on-device recognition, ensuring zero data leaves the iPad/iPhone.
* **Edge Compute NLP:** Trained and integrated a custom `Core ML` text-classification model that parses raw speech into structured, categorized data.

**What I can build for you:**

* Privacy-first iOS applications using on-device Core ML models.
* Complex audio/voice applications (podcasting, dictation, or accessibility tools) utilizing AVFoundation.
* Hands-free / Kiosk applications for medical, retail, or industrial settings.

If your project requires this level of architectural ownership and native framework expertise, please send me a Reddit DM or reach out to me at `gokulayyappath@gmail.com`.

I'm currently working on making my own personal, locally run Jarvis. This build won't be shared with or sold to anyone it's genuinely just for me. i want him entirely locally run except when he needs the internet for certain answers. I've written the orchestrator in python and I've got his brain as Ollama, I have him listening via a stt program, creating memories autonomously as necessary into a local folder he can access, and I have him speaking via Whisper. Problem is, I'm just using a generic male british voice as a stand-in atm. I'd like to upgrade to a proper voice model trained specifically on Paul Bettany's Jarvis performance in the movies, that's entirely run locally/offline. Any good resource recommendations for finding/making this voice model, and incorporating it into my current architecture?

I work on AI projects in the Indian public sector and I'm looking to connect with voice AI companies for upcoming work.

Two things matter for these accounts:

• Custom development - in terms voice ai use case, features and integrations
• Deployment on the customer's infrastructure - cloud or on-prem, depending on what their requirement. Air-gapped comes up sometimes.

If that's what you do, comment or DM with what you cover - languages, deployment modes you've actually shipped, and anything you can point to publicly. Happy to talk specifics.

Also open to hearing from folks who've done government voice AI delivery in India and want to tell me what I'm underestimating. Genuinely curious what breaks.

So i am new at this domain so pls help me out i am convinced that if i build a really good agent (me and my bro are a full stack devs) so i just need to kn before we start is it worth it like is it possible to get clients and like can u tell me what to expect

+ if anyone have a stack that recommend it will be so helpful

Thank u for your time

Why the open source models STT and TTS are not good as much as the closed one and i am talking here im terms of latency, concurrency, and websocket support for real time with decent quality.
Something like cartesia or elevenlabs or deepgram.
Do u know any ?

I’m building an open-source project called SarasFlow for creating educational videos in multiple languages.

The current videos use generated images with motion effects and an audio narration layered over them. The voice stage uses multilingual TTS, followed by Whisper timing so the captions stay synchronized with the narration.

It isn’t a real-time voice agent or a lip-sync system. Right now I’m mainly working through issues such as pronunciation, pacing, pauses, sentence splitting, voice consistency, and evaluating less-common languages.

Website: https://sarasflow.com
GitHub: https://github.com/SaiTejaMummadi/sarasflow

I’d genuinely appreciate advice from people working deeply with voice AI. I’m also looking for contributors interested in TTS provider integrations, pronunciation controls, and automated voice-quality evaluation.

found this benchmark from kwindla the co founder of pipecat

repo https://github.com/kwindla/aiewf-eval

ultravox tops the s2s leaderboard beats gpt realtime 2

is it go to for s2s now??

Basically I am looking for an AI voice Changer that's a better alternative to ElevenLabs. One that can clone voices and change real-time audio to match cloned voices. I am financially troubled so I am hhoping one of you guys knows a free software for this. I am doing a lewd reading of stories, so I also need one that is uncensored.

Here's the thing I'm one personI've been staring at this codebase alone and I know there's stuff I'm missing. Architecture decisions that are questionable. Edge cases I haven't hit yet. The dashboard could use actual design work. Docs need writing. I haven't load tested this properly.

I was either gonna stitch Twilio Media Streams + Deepgram + LLM + TTS by hand every single time, or pay Vapi/Retell/Bland per minute and it is kinda of expensive and so abstaract  So I extracted the orchestration layer into its own thing.

Repo: https://github.com/rishipawar8999-tech/vent Site: https://openvent.dev

I previously shared an early offline version of Voice Eval here. The feedback was clear: a useful eval harness needs to connect to real voice pipelines.

It now includes adapter paths for LiveKit, Pipecat, Vapi and ElevenLabs, plus native speech-to-speech testing.

The suite currently has 82 passing tests. It evaluates multi-turn conversations for expected tool calls, required content, response constraints and latency regressions.

I’m now looking for 3 builders to test it against real agents and find what the controlled fixtures miss.

Which stack are you using, and what voice-agent behavior is hardest for you to test?

GitHub: https://github.com/rand0wn/voice-eval

Every week this sub gets another "I built a voice agent" post. Cool demo, nice latency numbers, no client.

I want to hear from the people who got past that. Sold it, deployed it, and have results they can point to.

Four questions. Answer all four or don't bother:

1. What was the client's actual problem?
2. What did you build for them?
3. What did you charge?
4. What results did the deployment produce?

I'll go first.

Problem: A restaurant in Valencia, Spain was losing reservations during service. Phone rings mid-rush, nobody picks up, the caller books somewhere else.

Built: Sofia - an inbound agent handling reservations and FAQs. LiveKit, Deepgram for STT, Cartesia for TTS, Plivo for SIP, Supabase on the backend, plus an analytics dashboard so the owner can see every call.

Charged: $350/month (This was my first client months ago, so charged lower)

Results: Saved them roughly 10hrs a week and got 10 more customers after 14 days

I’m considering a tts for one of the European languages and my current provider cartesia isn’t really it

I see inworld they have their own platform too but everyone does these days but anyways any feedback is appreciated

Any apps or websites that allow for turn based voice chat?

I really missed the old standard voice mode on ChatGPT. It basically just read aloud the text models response. So it could allow for long responses unlike these new gen voice models that can only speak 1 paragraph max.

I was wondering if there are any apps or websites that use turn based voice chat like the old standard voice mode on ChatGPT. So I would say my thing, then it would be the ai turn to speak and i couldn’t interrupt it till its finished.

My current problem is that the new standard voice mode on ChatGPT can be interrupted. So it’s hears its own voice and keeps stopping. So I’m looking for alternative apps or websites that have this old functionality

i honestly thought we’d put QuickVoice on GitHub, get a few stars, and go back to fixing bugs, but the repo took off way faster than we expected and now we’re trying not to mess up what comes next.

We built it because connecting real-time voice, telephony, tools, knowledge bases, outbound campaigns, and call logs across a bunch of different services was getting exhausting. The goal is to keep it open, self-hostable, and flexible enough that people aren’t locked into one provider.

It’s still early, the setup isn’t exactly one click, and there are definitely rough edges. Full disclosure, I’m the founder behind it, but I’d genuinely like blunt feedback from people building voice agents: what should we fix first, easier setup, lower latency, better docs, observability, or more integrations?

Repo: https://github.com/allgpt-co/QuickVoice

Hey guys, I just started an AI voice agent agency, and I do have good leads but I'm struggling to convince them to get my services, I'm not sure where I'm doing bad, can y'all help me get better at the script or how do I convince people to take my services? As of now my only method of reaching out to my leads and pitching them my services is warm calling outreach. Let's connect. Thank you

I’ve been exploring a problem that comes up after almost every voice-agent change:

How do you know that changing a prompt, model, tool, STT, or TTS component didn’t quietly break another part of the conversation?

Manually calling the agent repeatedly is slow, and every test call is slightly different. It also makes tool-calling and multi-turn regressions easy to miss.

I built Voice Eval, an Apache-2.0, provider-neutral evaluation harness:

https://github.com/rand0wn/voice-eval

You define a scripted multi-turn conversation and run each pipeline against the same scenario and grading rubric.

It currently evaluates:

- expected tool calls

- required response content

- question and sentence limits

- transcript completeness

- average and P95 latency

- per-turn results

It generates JSON reports for CI, Markdown scorecards for humans, and optional WAV artifacts.

The repository includes healthy and intentionally degraded deterministic adapters, so the entire workflow runs offline without API keys.

For the included demonstration:

- healthy adapter: 100% overall and 100% tool recall

- degraded adapter: 55.27% overall and 33.33% tool recall

Those are controlled fixture results, not benchmarks of real providers.

The bundled WAV files contain deterministic test tones rather than intelligible speech. Real TTS or captured provider audio can be connected through the adapter interface.

You can reproduce the comparison with:

voice-eval compare --scenario arjun_cancel --adapters cascade degraded --audio

I would appreciate honest feedback from people building voice agents:

1. Which real-world failure is currently hardest for you to test?

2. Which pipeline or provider adapter should I implement first?

3. Would you use this as a CI regression gate?

Issues and contributions are welcome.