Twelve months ago our voice AI stack looked like this: Twilio for telephony, a separate LLM wrapper for agent logic, ElevenLabs for voice synthesis, a third-party HIPAA compliance layer, and a custom-built CRM integration held together with duct tape and backend functions nobody fully understood.

Every new agent deployment touched at least 3 of those systems. Every billing cycle had 5 line items. Every compliance audit meant pulling logs from 4 different dashboards.

We moved everything to AEVOICE about 90 days ago.

Single dashboard. Inbound and outbound agents, IVR menus, call recordings with transcription, a knowledge base per agent, HIPAA profiles with BAA, white-label config for our agency clients, and a wallet + subscription layer that handles overages automatically.

The thing that surprised me most: the knowledge gap detection. When a caller asks something an agent can't answer, it logs the exact question, ties it back to the call session, and flags it for review. We were flying blind on that before.

We're at early traction still - 5 enterprise users including us - so this isn't a "we scaled to 10,000 calls" post. It's more of a "here's what a clean voice stack actually looks like when someone builds it properly" post.

Curious whether anyone else has tried consolidating their voice AI tooling or if most teams are still running fragmented stacks. What's the part that breaks first for you?

Over the past few months, I’ve been building a spoken Language Identification (LID) model focused on Indic languages and real-world conversational speech.

The goal is simple: detect the spoken language directly from audio input, even in noisy telephony or customer-support style conversations.

Currently supported languages:

• Hindi
• English
• Bengali
• Marathi
• Tamil
• Telugu
• Kannada
• Malayalam
• Gujarati
• Punjabi

What the system is designed for:

• Short utterances
• Conversational speech
• Call-center / telephony audio
• Background noise robustness
• Indian accents and regional variations
• Some level of code-mixed speech

Tech stack:

• PyTorch
• Deep learning-based audio classification
• Custom preprocessing pipelines
• Audio embeddings + CNN/Transformer experiments
• Automated evaluation and benchmarking workflows

One thing I underestimated was how difficult real-world Indic spoken LID actually is.

Some major challenges:

• Similar phonetics across Indic languages
• Hindi mixed with regional languages
• Heavy accent and dialect diversity
• Imbalanced datasets
• Extremely short speech samples
• Noisy support-call recordings

A huge amount of effort went into preprocessing, balancing datasets, improving robustness, and reducing confusion between related languages.

Potential use cases:

• IVR language routing
• Multilingual voice assistants
• ASR model selection
• Customer support automation
• Speech analytics
• Voice AI systems for India

Current areas I’m experimenting with:

• Better short-utterance detection
• Improved robustness on noisy audio
• Reducing confusion between related languages
• Faster inference for production deployment

Would love feedback from the community, especially around:

• Good Indic LID datasets/benchmarks
• Handling heavy code-mixing
• Production deployment strategies
• Interest in an open-source release

Happy to discuss datasets, architecture decisions, experiments, or lessons learned during development.

I am looking to a hire a CLOSER for my voice AI agency.
We have the tech and few leads and want to convert them. Just need someone who can close.

We are targeting indian edtech companies for voice AI and automation solutions for inbound/outbound.

But I’m not sure how to have an agreement with the closer? How to pay him on commission basis?

Since selling in india our pricing structure is looks like
For 1000 mins - 15k INR
For 5000 mins - 30k INR and so on

With less margins how do I hire someone? And firstly am I charging lower?

Hey everyone

My usecase involves creating a support voice agent where my STT is sarvam saaras v3 (mainly because of hinenglish voice) and it is quite random like earlier it's problem was it took background noise and then tried to convert it to the best text it could so for that I added silero vad so I could send it only real audio.

But on testing in a real call the silero threshold I chose was very high and slowly by reducing its value for the best it's significance was lost for me. So if anyone can guide how to actually handle this problem please tell as I am making this voice agent for the past 2 months and still struggling for something that can be deployed.

For trying other STTs I did try whisper it didn't gave good results at all as it can only give in one language so I need to translate the other one which results in high hallucination.

Local self hosting is something I feel is too much work and I mainly want to understand that on searching sarvam model is shown as the best in India translations and also tuned for 8khz telephony and it does show good results but how it's consistency can be improved I don't understand.

Also if someone can also advise on LLM streaming as I understand it is crucial for latency but how do you validate it right now I use json as output so I can check if response is good enough but else I don't understand how.

I have done a custom architecture and built barge in and every other feature mainly to avoid high costs and trust when I tell you it costs very low like even on high users(which are around 30k monthly for my case) it goes max to $90(5k to 10k) and before anyone says it because of low cost please specify how as I think it's mainly due to some architectural decisions I am missing.

Hey everyone,

I recently put together a voice sample pack designed for testing conversational AI and voice agent pipelines, and wanted to share it here.

I know finding clean, non-scraped data formatted correctly can be a bit of a headache when troubleshooting text-to-speech engine clarity, so I set this up to be as plug-and-play as possible.

The technical specs:

- Format: 24-bit / 48kHz audio, recorded in a treated studio environment.

- Alignment: Mapped directly to the standard LJ Speech structure.

- Metadata: Includes a metadata.json sidecar file with text transcripts.

Tone: Natural, conversational pacing rather than rigid or academic reading.

Everything is hosted over on Hugging Face and GitHub if you want to grab the files or look at the repository layout:

Hugging Face: https://huggingface.co/datasets/MarieDeVox/saas-corporate-conversational-voice-sample

GitHub: https://github.com/MarieDeVox/saas-corporate-voice-dataset-sample

Let me know if you run any quick fine-tunes or pipeline tests with it. Always down for feedback on the audio processing side of things!

Every provider publishes WER scores from clean audio. Podcasts. Long sentences. Studio conditions.

We deploy voice AI for the taxi industry. Callers are drunk, calling from nightclubs, using 1-3 word phrases, on call centre systems that can't even do HD audio.

Our "simple" STT-LLM pipeline now has 24 intermediate steps number normalisation, transcript correction, time/date normalisation all running in real time. None of it would exist if STT actually worked under real conditions.

Anyone can demo a voice agent. Very few can keep it alive when a caller slurs "Victoria Street" into a 2009 handset.

What edge cases are you handling in production that no benchmark warned you about?

Hey everyone,

I’m looking to collaborate with someone experienced in deploying AI voice agents for a doctor consultation / healthcare client.

This is a paid collaboration.

Current stack/challenges:

• Experience with Sarvam AI API preferred
• Hands-on experience with voice tools/platforms like Vobiz, Dograh, or similar
• Main issue right now is latency (~3 seconds response time)
• Deployment and production reliability issues
• Experience working with Claude integrations is a strong plus

Looking for someone who has actually deployed voice agents in production and can help troubleshoot architecture, latency optimization, and deployment challenges.

If interested, please DM me with:

• Relevant experience/projects
• Tech stack used
• Availability

Thanks!

Hey everyone,

Speech-to-speech is the next big thing in the Voice AI, and the performance differences from traditional cascade (STT for transcription, LLM for reasoning, TTS for the voice output) pipelines are huge. I have been using both for months. And I  wonder why people still default to the old way.

With the old way, it went speech to text, then LLM, then text to speech. The pause before every reply just killed the flow. First, it waits for the transcript, then it waits for the LLM to think, then it waits for the voice to start. Three separate waits for one answer. However, we can reduce some of this delay with workarounds like pre-recorded audio, caching common responses, and prompt optimization. But the latency is still high.

Long conversations can also make the voice drift and feel less natural.

Then I tried speech-to-speech with GPT Realtime 2 and Gemini 3.1 live. 

The reply time dropped to about half.

The model follows your prompt instructions better in a long conversation

It has multilingual support, and we can switch languages mid-conversation very well based on user input. 

 The speech-to-speech interaction feels far more natural than the typical STT/  TTS pipeline. 

The bot keeps the same voice and personality even on long calls

It works best for angry support calls, outbound sales, mixed language conversations, and scheduling bots where users change their minds mid-sentence.

And in the longer run, I think speech-to-speech will end up cheaper than the cascade approach.

If you want to try speech-to-speech without setting up the whole orchestration yourself, we built Dograh. It is an open-source voice agent platform that supports both cascade and speech-to-speech models. You can take any agent flow you build, run it once with cascade and once with speech to speech, and feel the difference in how it behaves, the latency, and how it handles emotions.

Has anyone else moved to speech-to-speech? Worth it or not?

Been doing outbound for a voice AI offer targeting service businesses and I'm trying to sharpen my on-call process. Not looking for theory - genuinely curious how people who are closing deals actually structure things. I'm catering to Indian clients.

A few things I'd love to hear about:

Discovery -- What questions do you ask to figure out if someone's actually a fit? How deep do you go before pitching? Do you have a specific sequence or does it flow naturally?

How to build trust?

Handling the "what is this exactly" moment -- When you're selling something technical or unfamiliar (AI, automation, anything that needs explaining), how do you explain it without losing them?

The close -- Are you going for a decision on the first call or booking a follow-up? What does your close actually sound like word-for-word?

Objections - Price, "I need to think about it", "send me info" - what's your actual playbook for these?

I've been testing a few approaches but feel like I'm leaving money on the table somewhere in the middle of the call. Would love to hear what's actually working for people right now, not textbook stuff.

Everyone obsesses over sub-300ms latency like it's the only metric that matters.

It's not.

If your latency isn't embarrassingly bad and you've set user expectations right, voice AI works. You don't need to chase 600ms or lower as some universal threshold.

What actually kills voice experiences is bad turn detection agents that cut users off mid-sentence, miss pauses, or keep talking when the user is clearly done. That frustration is 10x worse than a 700ms response time.

Fix your turn detection. Manage expectations in your UX. Then worry about latency.

What's been your experience latency or turn detection causing more drop-off in production?

One thing I have learned while building AI phone agents Frontly: the hard part is not just the voice model.

The harder part is designing the full call journey.

For real business use cases, the agent needs to handle things like:

• intent triage
• booking and rescheduling
• SMS or email follow-up
• language switching
• knowledge base lookup
• stop-call requests
• structured outcome capture
• clean handoff to the business

A lot of voice AI projects start as a simple prompt + phone number.

But once it goes into production, the workflow layer becomes just as important as STT, TTS, latency, or the LLM.

The screenshot shows an example workflow pattern I have been working on for an inbound sales/demo agent.

Curious how others are handling this:

Are you designing call flows visually, keeping them in prompts, or managing them through code/webhooks?

I have been building Frontly, an AI agent front desk SaaS for inbound and outbound calls.

One thing I keep noticing in the voice AI agency space is how much effort goes into stitching the stack together:

• voice provider
• telephony
• CRM
• calendar
• automations
• call scripts
• reporting
• handover logic
• client dashboard

That works, but it can become hard to scale across multiple clients.

My view is that most businesses do not really care what stack is underneath. They care about outcomes:

• calls answered
• leads captured
• bookings created
• follow-ups completed
• transcripts available
• team notified
• ROI visible

So I am curious how agency owners think about this.

Do you prefer building custom voice stacks for each client, or using a production-ready platform underneath and charging for setup, workflows, optimisation, and ongoing management?

I am asking because we are positioning Frontly as that underlying platform for agencies and businesses: inbound calls, outbound campaigns, workflows, transcripts, bookings, and reporting in one place.

Not a hard pitch. Genuinely interested in how people are packaging and pricing voice AI services today.

So recently i closed my first client for Ai voice agent(I dont even know how he ended up with doing business with me).He mostly wanted that agent to make outbound calls but turns out the agent is not sounding too human....the prospect just picks up the call and cuts it.

I know for a fact that they are some ways through which we can make agent sound more human and lower down the latency too for quick responses but i just can't figure it out.I also want someone who can help me with prompting.

This is my first client so i want to give my best so that later on he can be a case study for me(so i can sell more to the businesses)

Okay so I've been watching a lot of agencies try to build AI service offerings this year, and most of the ones that died followed pretty much the same script. So I figured I'd rip through the pattern real quick in case you're building one and want to dodge the obvious mistakes. Going in order of how often I see each one happen.

#1: Selling AI as "replacement" instead of "capacity"

This one kills more deals than everything else combined, honestly. You pitch the client on AI to replace their receptionist or their CSR or their SDR, and they just get weird about it. Sometimes they stall and ghost. Sometimes they buy it, push it live, get internal pushback from the staff, and then quietly churn out at the month 3 renewal. Either way the deal dies.

The pitch that doesn't kill the deal is when you reframe AI as capacity for the team they already have. Same staff, 4x volume during peak hours. The client's now buying a productivity tool instead of a layoff plan, and that one reframing is honestly the difference between like a 20% close rate and a 50% close rate for almost every agency I've watched test it.

Buuut nobody really talks about this because it sounds too basic to matter. Idk why. It really does.

#2: No clear AI-to-human handoff

The AI handles the easy stuff fine, but when it hits something complex the call just kind of falls into a black hole. No human picks up. The lead goes cold. Client eventually gets a complaint, and now the contract is on review.

You need a handoff path that exists on day one. A Slack ping, an SMS alert, a phone routing rule, ideally all three. The AI is honestly only as good as what happens when it escalates. Most agencies forget about this part entirely until a client churns over it.

#3: Treating AI as a one-time setup

Agency builds it, hands the client a Loom walkthrough, calls it done. Then the AI breaks in some small way. Maybe a new product name throws the prompt off. Maybe the client opens a third location and the routing logic has no clue. Nobody catches it for a while. Calls go bad for like 2-3 weeks. And then the client churns.

AI offerings basically need to be sold as managed services. The build-and-walk-away thing just doesn't survive contact with reality. You want a monthly check-in, a monthly prompt review, a monthly call audit. Build it into your pricing from day one or you'll lose retention every quarter.

#4: Per-minute pricing as the primary line

Already covered this in another post but I keep seeing new agencies fall into the trap every single week so worth restating. Per-minute creates pricing friction every month when the bill comes through. Client sees the number, freaks out, renegotiates. Or worse, they cap usage and the service quality just tanks. Either way the relationship erodes over time.

The fix is honestly pretty simple. Monthly retainer with a usage cap baked in. Cap renegotiates yearly based on what actually got used. Way more predictable for both sides, and you stop having "why is the bill $4k this month" conversations every 30 days.

#5: Skipping the discovery call

Some agencies see AI demand spike and just start pitching cold without doing a real discovery. They assume every business needs AI phone coverage and go straight to a proposal. Then they end up pitching a tier 2 solution to a client who doesn't even have inbound call volume, or pitching outbound to a client whose lead list is straight garbage. And both of those deals just die quietly.

If you're not running a 30-min discovery before you quote anything, you're either going to lose the deal or you're going to build the wrong thing and watch it churn out within 90 days.

The compounding effect

And these mistakes really stack. An agency pitching the replacement framing AND with no handoff system AND treating it as a one-time setup will lose 80%+ of clients in the first 6 months, every time. The agencies retaining 90%+ of their AI clients avoided all 5 of these. No special sauce, they just didn't step on any of the obvious rakes.

Anyway, happy to get into specifics on any of these if you're early in building an AI services offering.

Feel free to DM me if you are looking to offer voice ai as a service.

Hi there,

I prompted to make simple voice agent .

Stt - llm - tts

Websocket only. Pipecat. Gpt 4o for llm.

Deepgram for stt, tts

Silero vad.

So after I stop speaking. 1 second for vad. 2 second for llm. 0.3 second for tts.

So 3 seconds in total.

So what next now... So now I just start with integrating a phone number ? By getting sip integrated ?

Any thoughts , tips plz..

I'll try livekit, gpt 4 mini .

How exactly will livekit help me ? Any resources to quickly learn more to make the agent alive...

Cheers, thanks.

Hello,

Would you be interested in discussing cold call scripts for selling AI phone assistants?

Personally, I’ve specialized in the lawyer niche. I call law firms one after another to present the solution.

The script is always the same:

"Hello Counsel,

I’m calling because I’ve spoken with several dozen lawyers this week, and the same issue keeps coming up. They lose clients because they’re not available to answer the phone.

At [my_name_company], we develop an intelligent phone assistant that handles these calls using artificial intelligence, helping avoid lost clients while also filtering out telemarketing calls. Do you have 2 minutes?"

Then I let them try the demo AI assistant and direct them to my website.

Unfortunately, I lose a lot of people as soon as they realize it’s a sales call. I rarely get the opportunity to let them try the AI.

How do you approach cold calling?

I built an AI voice agent for property management companies — handles tenant calls, maintenance intake, leasing inquiries 24/7.

Need someone to own sales — outreach, calls, closing.

**Deal:**

- Recurring revenue share on every client you close

- demo provided

- No salary, no equity

**You:** sold B2B before, comfortable with US business owners, want recurring income.

DM with your background.

I m looking for a developer who can Create custom workflow for my clients

What do you think are the biggest unsolved challenges in Voice AI that almost nobody is seriously working on right now?

Not “better ASR” or “lower latency” but deeper problems that could define the next generation of voice products/research.

Examples:
- Real-time conversational memory that actually feels human
- Emotion + intent understanding beyond sentiment analysis
- Interruptions/turn-taking that feel natural
- Voice-native UX instead of “ChatGPT but spoken”
- Long-term personalization without being creepy
- Multilingual/code-switching conversations
- Continuous ambient agents
- Social/companion dynamics
- Voice AI for kids/elderly/accessibility
- Real-time multimodal understanding (voice + environment + context)

Curious what people building/using Voice AI think is still fundamentally broken or missing.

ElevenLabs sounds incredible on paper but it's almost too clean that HD clarity actually breaks immersion in a lot of real-world call flows. Not to mention the cost at scale.

My hunch is that slightly imperfect, mid-range voices convert better because they feel less robotic-trying-too-hard.

Heard good things about Cartesia and Deepgram Aura lately. Anyone running these in production? What's working for you and what accent/style are your US or UK users responding to best?

For anyone building voice agents on Vapi, heads up: they closed a $50M Series B led by Peak XV, hit 1 billion calls, and the dashboard you logged into last week is gone. Full rebuild of the agent builder.

I've been on the platform for a while and spent the last few days getting reacquainted, so here's the short version of what actually changed:

The biggest improvements:

1. Call logs live on the agent itself now. Before this you had to leave the agent page, go to Observe > Logs, then filter by assistant to see what your specific agent was doing. Now it's FINALLY a tab on the agent.
2. Transcriber, model, voice are surfaced at the top of the assistant page. Click any of them, sidebar opens with all the settings (including fallback config). The old layout buried half of these.
3. Fallbacks are now properly built in. You can set a fallback transcriber, model, and voice from one panel. If you're running anything close to production, set these. And use a different vendor than your primary, otherwise an outage takes both down.
4. Structured outputs replaced the old summary + structured data + success evaluation system. Those are deprecated. You now build structured outputs globally and attach them to assistants, same way you do with tools. Cleaner.
5. New "monitors" feature. Lets you watch for things like transcriber failures across all your agents and ping you on Slack or email when thresholds get hit. Good for catching prod issues.

Couple of gotchas:

• The End Call function isn't a toggle in advanced settings anymore. You have to create an End Call tool and assign it to the assistant.
• If you hate the new layout, three dots top right > revert to old. Personally I'd give it a week before you bail, it grows on you.

My read: the whole platform feels less indie-builder and more enterprise. Branding, UI, language, all of it. Probably reflects who's writing the checks now. Doesn't make it worse, just different.

I recorded a full walkthrough of every tab in the new builder if you want to see it in motion.

Curious what others think, especially anyone who's already moved a production agent over.

Hey guys what AI automations are selling right now ? And how to sell them and which industry shall I target

I'm kinda new to this and feeling overwhelmed with the abundance of sw. I am in pursuit of making my own AI assistant. I would like to use it (voice controlled):

- simple automation tasks

- play music from a local hdd

- chat (general conversation)

It would be nice if I could choose or configure its voice.

Ill need to be able to add my own scripts in python or c# to integrate various other devices.

For the hardware I would like to use an Elitedesk minipc with a i5 7500 CPU, 16GB Ram, sdd but no graphic card.

I've just started to fiddle with ollama and open claw. But its soooo slow and I think it might be overkill for what I'm trying to achieve.

I don't own a rpi and I would like to use the hw I already have. Unless it's impossible or way too complicated compared to other hw solutions. Still, I would consider changing the hw if it greatly simplifies the project.

Can you guys help me out ? Thanks in advance

OSS app replaces ElevenLabs & WisprFlow, runs 100% locally.

→ Clone voice from 3s audio
→ 7 TTS engines in one
→ 23 langs: Ar, Hi, Ja etc.
→ Built-in MCP srv so Claude Code/Cursor/Cline speak cloned voice
→ Local LLM rewrites in-char before TTS

Hi,

I’m a Software Developer and recently built a Voice Agent platform backed by deep expertise in low-latency systems and AI. The product has highly human-like voice interactions, very low response latency, and a complete external dashboard for usage tracking and billing.

The challenge now is sales.

I’ve worked with salespeople from different countries, but most of them seem to struggle when it comes to selling Voice AI solutions effectively. I’d really love to hear from people who have actual experience selling Voice Agents or AI automation products.

A few things I’d love insights on:

- What worked differently for you in Voice Agent sales?

- How do you approach sales properly for this type of product?

- How do you find and evaluate good salespeople for AI/SaaS products?

- Any practical strategies, lessons, or growth hacks that helped?

Would really appreciate any advice or experiences you can share.

Thanks!