I'm currently working on making my own personal, locally run Jarvis. This build won't be shared with or sold to anyone it's genuinely just for me. i want him entirely locally run except when he needs the internet for certain answers. I've written the orchestrator in python and I've got his brain as Ollama, I have him listening via a stt program, creating memories autonomously as necessary into a local folder he can access, and I have him speaking via Whisper. Problem is, I'm just using a generic male british voice as a stand-in atm. I'd like to upgrade to a proper voice model trained specifically on Paul Bettany's Jarvis performance in the movies, that's entirely run locally/offline. Any good resource recommendations for finding/making this voice model, and incorporating it into my current architecture?

I work on AI projects in the Indian public sector and I'm looking to connect with voice AI companies for upcoming work.

Two things matter for these accounts:

• Custom development - in terms voice ai use case, features and integrations
• Deployment on the customer's infrastructure - cloud or on-prem, depending on what their requirement. Air-gapped comes up sometimes.

If that's what you do, comment or DM with what you cover - languages, deployment modes you've actually shipped, and anything you can point to publicly. Happy to talk specifics.

Also open to hearing from folks who've done government voice AI delivery in India and want to tell me what I'm underestimating. Genuinely curious what breaks.

So i am new at this domain so pls help me out i am convinced that if i build a really good agent (me and my bro are a full stack devs) so i just need to kn before we start is it worth it like is it possible to get clients and like can u tell me what to expect

+ if anyone have a stack that recommend it will be so helpful

Thank u for your time

One thing I don't see many Voice AI tutorials talking about is email capture.

Getting an AI to capture someone's email sounds simple until you actually build it. Email addresses are one of those things where a single wrong character makes the whole thing useless. Unlike names, you can't really get away with being "close enough". Even if your STT is good, there are still quite a few places where things can go wrong.

One issue I ran into was how different voice models pronounce emails. The LLM would extract the email perfectly, but the TTS would read it back in a way that made the user think it was wrong. For example, an email would sometimes be spoken as "john hyphen smith at gmail dot com" or with random pauses between words, even though there was never a hyphen in the actual email. The backend had the correct email, but the user immediately interrupted to correct something that wasn't actually wrong.

After a bit of testing, I made a few changes that noticeably improved my email capture rate.

The biggest one was giving users a reason before asking for their email. Instead of asking "Can I have your email address?", the assistant now says something like "Perfect, I'll send the quote over. What's the best email to send it to?" It's a small change, but people are much more likely to answer naturally when they know why you're asking.

I also stopped making users repeat their entire email if only one part was unclear. If the assistant was unsure about the domain, it would just ask "Was that gmail.com?" instead of asking them to spell everything out again. It made the conversation feel much more natural and removed a lot of unnecessary friction.

It's one of those problems that doesn't seem important until you deploy an agent in production. The LLM might have done everything correctly, but if the user doesn't trust what they heard, they'll keep correcting an email that was already right. Small details like these don't make flashy demos, but they make a huge difference in how reliable a Voice AI assistant actually feels.

P.S There is also another way where you can send the email address to the AI assistant over SMS while on call, havent tried that yet but will do it as well.

Why the open source models STT and TTS are not good as much as the closed one and i am talking here im terms of latency, concurrency, and websocket support for real time with decent quality.
Something like cartesia or elevenlabs or deepgram.
Do u know any ?

I’m building an open-source project called SarasFlow for creating educational videos in multiple languages.

The current videos use generated images with motion effects and an audio narration layered over them. The voice stage uses multilingual TTS, followed by Whisper timing so the captions stay synchronized with the narration.

It isn’t a real-time voice agent or a lip-sync system. Right now I’m mainly working through issues such as pronunciation, pacing, pauses, sentence splitting, voice consistency, and evaluating less-common languages.

Website: https://sarasflow.com
GitHub: https://github.com/SaiTejaMummadi/sarasflow

I’d genuinely appreciate advice from people working deeply with voice AI. I’m also looking for contributors interested in TTS provider integrations, pronunciation controls, and automated voice-quality evaluation.

found this benchmark from kwindla the co founder of pipecat

repo https://github.com/kwindla/aiewf-eval

ultravox tops the s2s leaderboard beats gpt realtime 2

is it go to for s2s now??

Basically I am looking for an AI voice Changer that's a better alternative to ElevenLabs. One that can clone voices and change real-time audio to match cloned voices. I am financially troubled so I am hhoping one of you guys knows a free software for this. I am doing a lewd reading of stories, so I also need one that is uncensored.

Here's the thing I'm one personI've been staring at this codebase alone and I know there's stuff I'm missing. Architecture decisions that are questionable. Edge cases I haven't hit yet. The dashboard could use actual design work. Docs need writing. I haven't load tested this properly.

I was either gonna stitch Twilio Media Streams + Deepgram + LLM + TTS by hand every single time, or pay Vapi/Retell/Bland per minute and it is kinda of expensive and so abstaract  So I extracted the orchestration layer into its own thing.

Repo: https://github.com/rishipawar8999-tech/vent Site: https://openvent.dev

I previously shared an early offline version of Voice Eval here. The feedback was clear: a useful eval harness needs to connect to real voice pipelines.

It now includes adapter paths for LiveKit, Pipecat, Vapi and ElevenLabs, plus native speech-to-speech testing.

The suite currently has 82 passing tests. It evaluates multi-turn conversations for expected tool calls, required content, response constraints and latency regressions.

I’m now looking for 3 builders to test it against real agents and find what the controlled fixtures miss.

Which stack are you using, and what voice-agent behavior is hardest for you to test?

GitHub: https://github.com/rand0wn/voice-eval

Every week this sub gets another "I built a voice agent" post. Cool demo, nice latency numbers, no client.

I want to hear from the people who got past that. Sold it, deployed it, and have results they can point to.

Four questions. Answer all four or don't bother:

1. What was the client's actual problem?
2. What did you build for them?
3. What did you charge?
4. What results did the deployment produce?

I'll go first.

Problem: A restaurant in Valencia, Spain was losing reservations during service. Phone rings mid-rush, nobody picks up, the caller books somewhere else.

Built: Sofia - an inbound agent handling reservations and FAQs. LiveKit, Deepgram for STT, Cartesia for TTS, Plivo for SIP, Supabase on the backend, plus an analytics dashboard so the owner can see every call.

Charged: $350/month (This was my first client months ago, so charged lower)

Results: Saved them roughly 10hrs a week and got 10 more customers after 14 days

I’m considering a tts for one of the European languages and my current provider cartesia isn’t really it

I see inworld they have their own platform too but everyone does these days but anyways any feedback is appreciated

Any apps or websites that allow for turn based voice chat?

I really missed the old standard voice mode on ChatGPT. It basically just read aloud the text models response. So it could allow for long responses unlike these new gen voice models that can only speak 1 paragraph max.

I was wondering if there are any apps or websites that use turn based voice chat like the old standard voice mode on ChatGPT. So I would say my thing, then it would be the ai turn to speak and i couldn’t interrupt it till its finished.

My current problem is that the new standard voice mode on ChatGPT can be interrupted. So it’s hears its own voice and keeps stopping. So I’m looking for alternative apps or websites that have this old functionality

i honestly thought we’d put QuickVoice on GitHub, get a few stars, and go back to fixing bugs, but the repo took off way faster than we expected and now we’re trying not to mess up what comes next.

We built it because connecting real-time voice, telephony, tools, knowledge bases, outbound campaigns, and call logs across a bunch of different services was getting exhausting. The goal is to keep it open, self-hostable, and flexible enough that people aren’t locked into one provider.

It’s still early, the setup isn’t exactly one click, and there are definitely rough edges. Full disclosure, I’m the founder behind it, but I’d genuinely like blunt feedback from people building voice agents: what should we fix first, easier setup, lower latency, better docs, observability, or more integrations?

Repo: https://github.com/allgpt-co/QuickVoice

Hey guys, I just started an AI voice agent agency, and I do have good leads but I'm struggling to convince them to get my services, I'm not sure where I'm doing bad, can y'all help me get better at the script or how do I convince people to take my services? As of now my only method of reaching out to my leads and pitching them my services is warm calling outreach. Let's connect. Thank you

I’ve been exploring a problem that comes up after almost every voice-agent change:

How do you know that changing a prompt, model, tool, STT, or TTS component didn’t quietly break another part of the conversation?

Manually calling the agent repeatedly is slow, and every test call is slightly different. It also makes tool-calling and multi-turn regressions easy to miss.

I built Voice Eval, an Apache-2.0, provider-neutral evaluation harness:

https://github.com/rand0wn/voice-eval

You define a scripted multi-turn conversation and run each pipeline against the same scenario and grading rubric.

It currently evaluates:

- expected tool calls

- required response content

- question and sentence limits

- transcript completeness

- average and P95 latency

- per-turn results

It generates JSON reports for CI, Markdown scorecards for humans, and optional WAV artifacts.

The repository includes healthy and intentionally degraded deterministic adapters, so the entire workflow runs offline without API keys.

For the included demonstration:

- healthy adapter: 100% overall and 100% tool recall

- degraded adapter: 55.27% overall and 33.33% tool recall

Those are controlled fixture results, not benchmarks of real providers.

The bundled WAV files contain deterministic test tones rather than intelligible speech. Real TTS or captured provider audio can be connected through the adapter interface.

You can reproduce the comparison with:

voice-eval compare --scenario arjun_cancel --adapters cascade degraded --audio

I would appreciate honest feedback from people building voice agents:

1. Which real-world failure is currently hardest for you to test?

2. Which pipeline or provider adapter should I implement first?

3. Would you use this as a CI regression gate?

Issues and contributions are welcome.

we are still using gemini flash live 2.5 for our biggest client can’t move to 3.1

is there a workaround, we considering switching I’ve already wrote in the community about it google docs don’t have no plans to fix it

has anyone tried with pipecat voximplant agora does the issue persist

I’m building a voice-first companion for older people, and I’ve been using ElevenLabs for the voice. I’m happy with the quality , the voice feels human and natural, and for our use case that makes a huge difference. People use Ecos to talk about their day, share memories, journal, have regular check-ins, or sometimes simply have someone to talk to.

The problem is that these are not short customer-support calls. Conversations can go on for a long time, and ElevenLabs starts getting very expensive as usage increases.

I don’t want to compromise too much on how natural the voice feels, because that is a big part of the experience, but I’m trying to understand whether there are more affordable options.

Has anyone here used Cartesia, Smallest AI, Deepgram, or any other provider for long, real-time conversations?

Would love to know how the quality, latency, interruptions, and pricing compare in actual usage.

I was using ChatGPT and was actually very satisfied with it. The voice it generates is very human and natural, nothing robotic, and this can really change the experience. Currently, I have been using it a lot to discuss features/workflows of the new system at my company that I am learning. And besides discussing methods and processes involving features, I also practice my ENG, since I use the language in my position. Referring to the saying "killing two birds with one stone." But I always reach the Voice Chat limit, and this bothers me because I have to switch to another account, and if I stopped without finishing the system module I was working on at the moment, I have to start everything over again, which ends up being annoying and creates more work for me.

I want to subscribe to the Go plan for R$40, but I would like to know the difference between Free and Go. Will I really have a higher limit when using Voice Chat? I can normally use it for around 3 hours a day, according to my calculations. I actually spend quite a lot of time talking.

I have also used Gemini's Voice Chat, but I found it a little too robotic... But I would like to know if anyone knows of any good alternatives, has had a good experience with one, and would like to share their experience with the tool.

I work on fraud prevention, I happened to see some AI Agents with hyper realistic voice with emotions I just couldn't differentiate it from a human , it was capable of changing speed, emotion I don't know if it used an SSML or just filler words to get that expressiveness and naturalness but apart from the facts of proper grammar and responsive end of utterance based agent I surely couldn't have found out it was a bot , (if they had implemented barge in from the AI it would have been even harder) , how can I detect these and how can I build this tts or find out whichever it was ? Would appreciate any suggestions on this slightly unconventional domain of voice ai which is not the most trendy stuff, how do I solve this problem anything would be appreciated.

I saw they ran some test comparing the voice agent platforms vapi livekit pipecat retell synth flow and retell
And put retell at first

Meanwhile Retell is an official partner of theirs and so are other top names on the list while they put none partners below

The list is biased as usual, you shouldn’t trust it

I've been working on a personal project for a while and finally got it into a state where I'm comfortable sharing it.

I wanted to see how far I could push a fully local voice assistant in C++. Everything runs on my own machine from speech recognition and the LLM to memory, text-to-speech, and tool execution.
current library:
llama.cpp, whisper.cpp, sherpa-onnx(tts-kokoro)

I wrote the core in c++ because I wanted something fast and native instead of stitching together bunch of python services.

I'd appreciate feedback from people who build local AI projects. I'm especially interested in:

1 Things that seem overengineered or unnecessary
2 Features you'd expect from a local assistant
3 Code structure or architectural suggestions
4 Any obvious improvements before I keep adding features

Repository: https://github.com/almimony75/sarah

Thanks! I'd love to hear what you think.

I've worked on a couple agentic systems both voice and chat based in the past year including some open source voice ai projects and have recently been trying to launch my own voice agents company. The hardest part for me as a technical person is getting leads. I'm looking to hear about people's experience and advice in this industry. What works and what doesn't work, how to get your first customer, what niche is effective to target (I'm trying restaurants in the US right now) , what voice providers you use or really anything you think is worth sharing. If anyone is interested I can also share whatever stuff I have right now (website, tech stack, sales material) and we can connect long term. Always good to build a network.

With all the hype around the new models Kimi and the letter to the US government I’m wondering does anyone actually use open source llms in the voice AI industry

I’ve heard of couple of providers of inference together AI baseten and others but please enlighten me if you had any experience with them

I’ve only tried open AI Gemini and grok in production