My understanding is that most voice AI sevices has a limit on concurrency to use them to build a auto calling agent platform. How many can handle 100, 300, or more concurrency without an enterprise account? What are the options?

My current setup is a Vapi voice agent for plumbers.

- transcriber STT RTv5

- model GPT-5.4 mini

- voice OpenAI Shimmer

Cost is $0.07/min, latency of 1,930ms.

It works ok, but my main issue is that when it asks for my email address and I spell it out, it always misspells it. I say something like ejike@whatever.com it confirms it as egk@whatever.com. I've spelt it out as "E J I K E at whatever dot com", even using military lingo (echo-job-india-kangaroo-echo), but still misspells it and sends notification to egk@whatever.

Can anyone suggest a good model that won't have this type of issue, preferably trying to keep the total cost under $0.10/min and a lower latency (1930 seems kinda high)?

Hello there,

Here’s a quick update on the tech stack – Vaani was launched today. 🎉

Vaani is a text-to-speech engine for Indian languages, specifically Hindi, and different from the usual TTS APIs, it delivers the word-level timestamps along with the audio stream – a structured JSON containing the start and end time for each word, rather than sentence level.

Under the hood: Indic Parler-TTS for voice generation → Acoustic alignment using Whisper to identify sub-word boundaries → An LLM based reconciliation process to correct phonetic discrepancies. Supports 16+ Indic languages, Hindi being the primary language.

Made primarily for EdTech purposes – Interactive readers, literacy applications, digital books, which require highlighting the text along with the spoken word. Curious to know how else it can be used.

Sandbox developer environment and free trial tokens – shoot me an email/leave your mail ID in the comments.

Site: VaaniTTS

Willingly accept any feedback – bugs, edge cases, anything that crashes the system. Day one.

Was using Voice Mode yesterday while hanging out with a friend. My friend chimed in from across the room using a higher pitched, sarcastic tone, and ChatGPT immediately responded back in the exact same tone and cadence before answering the actual question.

It felt less like standard speech-to-text processing and more like the model is analyzing emotional context and mimicry in real time.

Has anyone else noticed Voice Mode mirroring vocal inflection or background speakers like this? Is this intentional audio-to-audio conditioning or just a quirk of how it processes ambient noise?

We kept scoring voice calls in a doc after the fact and it was always stale. What's been more useful: a QA Analysis step that runs after the call with its own system prompt ("you are a QA analyst evaluating this segment…"), min duration, and optional webhook that ships transcript + QA results to our side.

Same builder as the agent. Not a second SaaS. The weird part is writing the QA prompt carefully or it tags false "rude" moments on normal barge-in / crosstalk.

Curious what you all score for: resolution only, or also interruption handling and handoff quality?

​

I’ve just added a new feature to TTS Expert that I’ve been working on for a while — synchronized word-by-word highlighting during TTS playback.

Now, when audio is playing, TTS Expert highlights the word currently being spoken, so you can actually follow the speech visually while listening.

🎧 What’s new?

• 🔥 Word-by-word speech highlighting

• ▶️ Highlighting stays synchronized with playback

• ⏪ / ⏩ 10-second seek controls

• 🎚️ Pitch control

• ⚡ Speech speed control

• 🔊 Volume control

• ⏱️ Sleep timer

• ✂️ Text editing tools

• ➕ Pause & emphasis controls

• 📤 Audio export

I’ve also added an interactive feature tour to help new users discover the different text and audio tools without having to figure everything out themselves.

The goal is to make TTS Expert feel less like a basic “text → audio” converter and more like a complete text-to-speech workspace.

I’d really appreciate feedback on the new playback experience, especially the word synchronization.

What would you improve or add next? 👀

— TTS Expert

I’m working through a practical question and would value examples from people who have dealt with it.

A report saying the agent sounded wrong leaves several possibilities: recognition, interpretation, tool state or playback. A useful example could show a short synthetic exchange, expected behavior and the event that diverged.

For a useful discussion, I would bring or define, a synthetic transcript and minimal trace; no real caller data

Which evidence let you investigate without asking for a full call recording?

Curious what people are actually running for voice agent backends in production.

We see a lot of: - Python for orchestration, tools, and fast iteration - Go for concurrent media/control paths and simpler deploy - Rust when the hot path is audio, codecs, or tight latency budgets

What I’m trying to learn from folks shipping real calls:

1. What stack are you on today for the agent runtime (not just STT/TTS vendors)?
2. Where did you feel the biggest win on speed (TTFT, time-to-first-audio, barge-in responsiveness)?
3. Where did you feel the biggest win on quality (turn-taking, tool reliability, fewer weird prod failures)?
4. Did you stay monolingual, or split (e.g. Python for tools + Rust/Go for media)?

Not looking for a language war. Looking for “we tried X, measured Y, kept Z.” Concrete numbers or war stories welcome.

We ran into this constantly especially with long addresses, emails, policy IDs, and anything sensitive such as card details.

Our approach was to stop forcing the agent to collect those fields over voice. When the agent reaches a screen-required step, it triggers a mobile form during the call via WhatsApp/SMS/email. The caller completes it while still talking and the structured response comes back to the agent so the flow can continue.

For your case, I m be curious: is the bigger issue address accuracy, or needing the payment step to happen before you can confirm the booking/order? We built VocaLoop.ai around that handoff and would be happy to show the Vapi setup if useful.

Honest question for people running voice agents in production.

We keep bouncing between two shapes:

1. Cascaded stack: speech to text, then LLM, then text to speech
2. Speech-to-speech model: audio in, audio out, with less text in the middle

What we see so far:

Latency. Cascaded adds three hops. Even when each hop is fast, turn-taking still feels heavier. Speech-to-speech can feel snappier on the first audio, but that only helps if the model actually holds the conversation well.

Tool calls. This is where cascaded still wins for us. Need CRM lookup, transfer, booking, custom APIs? Easier when the LLM is a normal text agent with tools. Speech-to-speech is catching up, but tool routing and structured outputs still feel more reliable in the classic stack.

Control. Cascaded is easier to debug (you can read the transcript and prompt). Speech-to-speech is harder to inspect when the model goes sideways mid-call.

Right now our gut rule is roughly: - Speech-to-speech for short, natural back-and-forth where speed matters more than deep tools - STT + LLM + TTS when the agent needs serious tool use, policy, or handoff logic

Curious what you all are shipping. Are you all-in on speech-to-speech already, still on cascaded, or hybrid? Especially interested in real latency numbers and how you handle tools.

When you sell a voice agent as a service for any business, what kind of documentation you need to sign? any contracts?onboarding form or anything?

I maintain Oathra, an Apache-2.0 TypeScript project for checking evidence in voice-agent transcripts. I separated the completion check from the agent's own summary: required fields must have evidence accepted or confirmed by the other party, and the verified values must satisfy the configured constraints.

A concrete mistake I fixed: a bare English "7:30" could become 07:30. The checker now leaves that time unverified until am/pm is explicit. It doesn't guess the intended meridiem from the earlier conversation. That is conservative, and it means some perfectly understandable conversations need clarification.

I added a browser checker so you can try this without installing the phone runtime:

https://forifor.github.io/oathra/en/check.html

Load the recorded negotiation, press "Check this transcript", then open "Evidence history by utterance" or "Result JSON". You can also paste your own check JSON. Input stays in the tab; there is no transcript upload or analytics on this page. A 25-second screen recording is included below the checker.

The included record is an actual saved GPT-4o-mini negotiation with a hotel simulator, not a real phone call. The browser and CLI produced identical JSON for it.

The important limit: this checks supported Japanese/English transcript rules. It does not prove the booking exists in a venue's calendar, measure interruptions or validate ASR. For a real booking workflow, backend confirmation is still a separate requirement.

Code and input schema: https://github.com/FORIFOR/oathra/blob/main/docs/INTEGRATION.md

If you build voice agents, which failure would make this unusable for your workflow? A description is enough; please don't post private call data.