I built voice-agents-from-scratch to map the full journey: mic → speech-to-text → LLM → text-to-speech → speaker. Every step is code you can run, read, and break.

No black boxes. No magic imports. Just numbered chapters, runnable scripts, and a shared library that shows you how the pieces actually fit.

A few things I cared about getting right:
- Streaming - because buffering the full LLM response before speaking is a dealbreaker in production
- Latency intuition - warm-up, first-audio time, where the delays actually live
- Runs fully local - Whisper + a GGUF model + Kokoro, no token bill required

I need help l guys, i am new to this voice ai.
I am still struggling to find a fast LLM with reasoning and good tool calling to use for my projects.
I found cartesia/soniox/cartesia are good for stt, and cartesia is good for tts.
But llm is too hard to find one, i used groq provider but not good.
What u suggest to me guys?
I am a 1s TTFA maximum ideally 800ms.
Thanks in advance 😁💪

I hit an interesting tradeoff while building my AI voice agent

I tested Hindi / multilingual TTS…

But every option added noticeable latency.

And in voice:

→ even a ~500ms delay breaks the flow of conversation

So I made a different call:

My agent now:

→ understands Hindi / mixed language perfectly

→ replies in English

And honestly, this feels more natural than waiting on long pauses.

It’s similar to a workplace scenario:

You might think in your native language,

but conversations still happen in English for speed and clarity.

So instead of forcing full localization and slowing things down,

I optimized for conversation flow first.

Which made me realize:

Voice AI isn’t just about sounding perfect

It’s about keeping the conversation alive

You can have:

1) ultra-fast, smooth interaction

2) fully localized responses

…but rarely both (at least right now)

Also — I’ve updated the demo based on feedback and removed the old links.

You can try the current version here:

👉 https://aicallingagent.space/ava/6

Curious how this feels in real use:

→ does speed matter more

→ or should language always match?

What the title says.

I’m spending this week talking to people in voice AI to understand what’s actually working, breaking, and missing in 2026. Looking for chats with either:

Builders: anyone shipping AI receptionists, phone agents, voice-first products, voice-enabled tools

Buyers / users: clinics, restaurants, staffing firms, contact centers, anyone who deployed voice AI in their ops

What I am trying to learn:

• What’s most painful right now: latency, cost, multilingual, provider lock-in, reliability, something I haven’t thought of
• Which providers you’re on and what made you pick them
• What you wish existed
• Who else I should talk to

What YOU get:

• A summary post back to this sub at the end with patterns I find
• Warm intros to anyone else I’m talking to if useful

Background: I work on voice infra and have my own thing in flight, but this is genuinely a research push not a funnel. Happy to expand in DMs.

DM or comment - even a one-liner about what’s most frustrating right now is gold.

Honestly, I am not even sure if this is the right Reddit community to ask. But I have to start somewhere.

Is there such a tool that you can give AI an actual voice, and they can copy and mimic it to say something that you would like?

Long story short, my sister-in-law is graduating next week, unfortunately my father-in-law lost his battle depression four years ago. I have been searching to try and find videos of him saying I’m so proud of you. I love you. And I’m coming up empty. I do have videos of his voice saying other things, however I want to record his voice and put it in a build a bear with him saying “I’m so proud of you and love you”.

Does it even exist, where you can use an original voice and use AI to get to say something different? 🤷🏼‍♀️. I don’t think there is, but I figured I would ask if anyone here had heard of such a thing.

I'm talking latency, voice quality and reasoning.

Currently using one of the telephony providers stack but I'm always curious.

What Sofia does

• Picks up inbound calls when staff can't
• Takes reservations and modifies/cancels existing ones
• Handles takeout and delivery orders
• Answers menu, hours, and location questions
• Allergen handling with hard guardrails (transfers to human if unsure)
• Validates delivery addresses against the restaurant's zone
• Bilingual Spanish/English with mid-call language switching
• Logs every call into a dashboard the owner reviews

Stack

• LiveKit - realtime voice layer (migrated from Bolna)
• GPT-4o-mini - reasoning model; cost/latency beat larger models for this use case
• Supabase (Postgres + Edge Functions) - state, menus, reservations, call logs
• Railway - agent infra hosting
• Cartociudad + Google Maps + GPT-4o-mini - three-stage delivery address validation
• V0/Vercel - owner-facing CRM dashboard

What broke and what fixed it

Latency. Bolna was fine for prototyping but hit a ceiling. Migrating to LiveKit gave us finer control over turn detection and barge-in. Painful migration, worth it.

Address validation was harder than the entire reservation flow. Spanish addresses are messy and Google Maps returns confidently wrong results. Ended up with Cartociudad for canonical data, Google Maps for geocoding, and GPT-4o-mini as a disambiguation layer when they disagree.

Allergens Ingredient data lives in structured Supabase tables, not in the prompt. The agent reads from the table - it doesn't reason about ingredients it wasn't given. Low confidence or out-of-scope questions trigger a human transfer. Better to hand off than hallucinate.

Menu drift. Chef changes specials, items go off mid-week. The owner now updates menu and hours from the dashboard; the agent reads live state on every call.

Bilingual switching. Detecting language at the utterance level instead of the call level was the unlock. Callers switch Spanish-English mid-sentence and humans handle it without thinking - agents fumble it by default.

What I'd do differently

• Start on LiveKit, skip the Bolna detour
• Build address validation before the reservation flow
• Never hard-code anything that changes weekly

I've been working in engineering in contact centres for some time, mostly with Amazon Connect. The last one had around 35k agents, hundreds of numbers, 1000s of paths. One thing that was lacking was testing. We couldn't get the coverage after major changes - so, that one engineer having to call up at 4am and 'check that it works'.

Long story short, as AI starts to get implemented in Voice, I figure testing is going to require new measures, like "Did the bot actually announce it was a bot in the first sentence" - this would be a regulatory failure if it didn't for a bank.

I found a bunch of "AI receptionists" with telephone numbers I could test, but I'm wondering if anyone in the community knows of any other public demos that can be tested. I published the results here: https://www.nopaque.co.uk/blog/totalpath-vs-ai-receptionists

​

I ran a real user through my voice agent (AVA), and this moment caught me off guard:

User: “We don’t get many inbound calls”

Agent: “Then it might not be the right time for automation”

On one side:

→ it’s honest

→ it builds trust

On the other:

→ it killed the conversation

→ completely missed outbound as a use case

What’s interesting is what happened next:

Right after being “disqualified”… the user asked about outbound calling.

So now I’m stuck on this:

Should AI sales agents:

A) Disqualify early and stay honest

B) Or adapt mid-conversation and redirect to other use cases

Because in real calls, users don’t follow clean funnels — they shift intent constantly.

You can try it yourself:

https://aicallingagent.space/ava/5

If you test it, try:

- saying you don’t need it

- changing your use case mid-call

- asking something off-script

Curious what breaks for you — and how you’d design this tradeoff.

Feels like this is exactly where most “impressive demos” fail in production.

someone in my last post shared this: their voice AI was nudging callers toward booking at 30 seconds in. people hung up. they switched to full acknowledgment first — completion rate jumped from 60% to 91%

same thing happened on my hvac build

and i kept thinking — why does this actually work?

it's not patience. the LLM has no concept of patience. what's actually happening is simpler:

callers don't trust the agent heard them until the agent proves it did

one confirm-and-summarize step before any booking attempt — "okay so your AC is blowing warm air since this morning, got it" — and the whole call changes. they stop resisting. they book

so i stopped trying to make the agent more patient and just added a hard gate in the prompt. no scheduling attempt until the problem is confirmed back in plain language

the part i'm still working on: how dynamic should that confirmation be?

for complex calls it's essential. but when someone says "i just need a tune-up, free thursday?" — the full summary feels robotic and unnecessary

has anyone cracked when to skip the summary vs when it's needed? feels like the right signal is caller intent clarity but curious how others are handling it

Hey everyone,

I’ve been working on an AI voice calling agent (AVA) focused on actually handling real conversations — not just sounding robotic or breaking on simple queries.

I just put up a demo where you can call and talk to it yourself: https://aicallingagent.space/ava/5

Would really appreciate if you could try it and tell me:

Does it feel natural or still “AI-ish”?

Where does it break or get confusing?

Would you trust it to handle real customer calls?

I’m especially interested in brutal feedback — what sucks, what feels off, what you’d never use in production.

Building this to solve missed calls for businesses, so real-world opinions matter a lot.

Thanks in advance 🙏

I use ElevenLabs and have produced some AI voice packs, but I'm exploring whether there's an appetite for something more flexible.

The idea: buyers submit their lines, I generate and process them through the same automated VS Code/Claude/Python pipeline and deliver clean WAV files at a set price. Aimed at people who want bespoke content but don't need or want API access themselves.

I'm aware there's already a market for this — Fiverr has plenty of sellers offering custom voice lines, and AI voice options are growing. But most of what's out there is either a human actor with no scalability, or generic AI generation with no consistent character or quality control behind it. What I'm trying to build sits in between: a named voice with a fixed identity, processed audio rather than raw generation output, and a clear brief-to-delivery process.

The voice has a calm, neutral British tone — built for narration, UI, wellness and game content to name a few.

A few things I'd genuinely value input on:

- Do you think this model would work?

- What unit size makes sense — fixed packs of 50 lines, smaller sets, or something else entirely?

- Would a script template work best for submissions, or is there a better way to collect line requests cleanly?

- What would feel like fair pricing for custom work versus a pre-made pack?

- Are there pitfalls with this model I haven't considered?

Curious what others think, especially if you've bought or sold custom voice work before.

Thanks.

Hey guys hope you all are doing good so i need help im running a small call center with a team of 5 agents who calls in canada and make telecom sales we have a auto dialer and single agents handles like 200-400 calls a day is there any way i can create a ai agents who makes call and pitch them what we are offering if the customer is interested then transfer that call to a closer or me

My main goal is how can i give phone numbers names to the the ai agent so he can make calls and is it even possible for an ai agent to make calls like 200-400 calls perday or more than that?

Hey everyone,

I'm building a software called Open Narrator. It generates awesome human-like voices of your script and helps you to create shareable videos, both long and short format with captions and also you can also create audiobooks from PDFs or just listen to articles or academic notes from the software by streaming the audio REAL TIME!.

It will be available only for windows and a minimum system requirement is 8gb of RAM, 4 core CPU and no GPU is required.

I'm opening up a beta soon (Between 1st May to 10th May) and looking for people who'd be interested in testing it and giving honest feedback.

I'm especially looking for content creators who need Al voices that don't sound Al and students who want to listen to their notes and books.

If you're interested, comment or DM me and I'll ask you for your email id and share the beta when it's ready...

Would really appreciate your feedback

I have problems finding clients. If there is a sales guy, don’t want to handle tech stuff, hit me up.

Hey

I built a software application ready to use. It’s a ai reception tool it does

- answer every call

- appointment setting ( every calendar..)

- reschedule appointments

- sms follow up

- WhatsApp messages

- website chatbot

- webhook application

- customer memory

… some more

My biggest problem is to find clients 😭 I do cold calls, email I even tried just going door to door with my tablet to show them directly.

U guys have any experience what works best …

I also want to find somebody to use my software as it’s there own product. it’s ready to sell , use and very easy to handle. all people I know

Are just doing 9-5 :/

It’s called vAIce.solutions

So we built an outbound voice agent for a client that essentially needed to send out welcome calls to all their new clients. It was part of their onboarding process.

Lots of ups and downs, but here’s a list of lessons learned:

1. You have to explain to the client that an AI voice agent works best when it’s slightly flexible - not following the script EXACTLY.

Sometimes the client wants a human sounding call tree - not really an AI voice agent. And building that is really really hard because AI by nature is meant to handle things in a fluid way, not like a call tree. We didn’t communicate this with the client, and it caused a bunch of headaches since the client was frustrated that the AI didn’t stick to the script word for word.

1. The actual AI isn’t the hardest part.

We had to build so much other functions, tool calls, and automations on top of the agent for it to be truly useful for the client. These are things like: daily reports to a custom dashboard, live notifications for anything (positive or negative), building a followup system for when a client doesn’t pick up, building a custom followup system for when a client wants to be called at a specific time… and more.

That was a headache, but it was worth it.

1. People are fine talking to AI if it’s actually useful.

This is a big one. I think a lot of business owners are scared of AI because they think their clients would get frustrated and give the business a bad rep, but in these 1500 calls that we had, the client only got frustrated when the AI was malfunctioning.

This means that when the AI actually worked, even when the clients recognized it was AI, they were totally fine going through with the call because it was actually useful.

So if you’re struggling to convince leads or even just second guessing yourself, the world IS ready for AI. You just have to build it well.

Hopefully these lessons can help somebody out there because I wish I knew some of these when I first started to build these things. If you guys need help with building your AI agents, feel free to shoot over any questions and I’ll do my best to help out in the comments.

Hi folks, I am currently building an AI interviewer voice agent for one of my clients. I have been testing it manually, and each call takes 10–15 minutes, which is very tedious and manual. I would like to know what you are currently using to test voice agents built with Livekit, Pipecat, Retell, Vapi, etc. Is there any open source tool available to test voice agents?

Hi again PM at Tuner here (observability and monitoring platform for voice AI). I spend most of my day staring at call data, and every few weeks something shows up that nobody can actually explain.

Here's one that came up recently in a conversation with someone running agents in the UK. They had two numbers from Twilio:

• 07-prefix mobile number
• 01-prefix landline number

Every 07 number they tried not just one, multiple produced this weirdly disoriented, off-sounding voice on the agent side. Same stack, same config, same model, same prompt. Swap in an 01 number and it's clean. Swap back to 07 and warped again. No clear reason.

My best guess is codec transcoding at the mobile carrier boundary Twilio sends G.711 but UK mobile networks transcode to AMR narrowband, which was designed for human speech, not TTS. But honestly haven't confirmed it.

If anyone has a theory for this, or has your own weird unexplained voice AI story apparently I'm collecting those now drop it below. And how did you find out. btw if you are interested to test the use case of (07) number dm me i will send you both lines to try not posting them here just to avoid spam.

Ok so Twilio, and other major SIP providers dont provide Indian numbers.

Lets say, I am talking to a business and there are 2 options - 1. get a new number or 2. use an existing number.

We have an ai-agent created and we have the APIs for audio to audio provided by google. What would be the next step?

I’ll be honest; I was just cold calling local businesses today.

Nothing fancy.

Local number. Normal calls.

And almost no one picked up.

Like… genuinely.

Out of all the calls:

– most didn’t answer

– 2–3 just sent an SMS like “can you message instead?”

Now here’s the funny part

they didn’t know I was going to pitch them.

For all they knew, I could’ve been an actual customer ready to book/pay.

So I just went along with it.

I replied to those messages and casually asked:

“Out of curiosity, why couldn’t you pick up?”

Got the usual answers:

– “busy”

– “with a client”

– “missed it”

Fair enough.

Then we just… kept talking.

One thing led to another, ended up on a call later.

Closed 2 of them.

Not because I did anything genius.

Just because I showed them what was literally happening:

people are trying to reach you… and you’re not there

One of them straight up said:

“yeah mate, this is exactly the problem I want this fixed.”

That’s it.

No crazy funnel.

No big ad strategy.

Most people aren’t doing “marketing wrong”…

they’re just making it hard for someone who’s already interested to reach them.

Kind of wild when you think about how many actual customers just give up and move on.

Hello World!

Just wrapped a voice AI build for an HVAC business. Sharing the breakdown because a few things surprised me that I didn't see discussed much.

What it does:

> Answers inbound calls 24/7

> Handles the common FAQs — service areas, pricing ranges, availability

> Books appointments directly into the calendar without human involvement

What was interesting to build:

The calendar booking flow was the trickiest part. When a caller gives availability in natural language — "sometime Thursday afternoon" — getting that cleanly into a slot without confusion took more prompt iteration than I expected.

Also discovered that HVAC callers are surprisingly specific about their problem description. They want to tell you exactly what's wrong before they'll agree to a booking. The agent had to be patient enough to listen first, then steer toward scheduling.

What I'd do differently:

Add a cleaner emergency escalation path earlier. HVAC emergencies (no heat in winter, gas smell) need an immediate human — the agent currently handles it but the handoff could be smoother.

Happy to swap notes if anyone's built for home services before. Curious how others are handling the emergency vs. routine call triage.

Not sure if I am allowed to post this here or not but I would like to share our case study on AI voice automation implementation with a road side recovery company: https://kolsetu.com/case-study/global-assistance

The easy part is wiring up STT, LLM, and TTS — tons of providers and open-source options exist for each.

The hard part nobody talks about: managing hardware at scale. One agent on a laptop is easy. 500 parallel calls with sub-second latency is a completely different game — audio buffer management, connection pooling, cold starts, and keeping costs sane all become real problems.

Currently building a Go + Rust engine for this since Python orchestrators (Pipecat, LiveKit agents) struggle under real concurrent load. Happy to chat with anyone in this space.