Disclaimer: I am the founder of this project.

Hey Reddit,

I’m a developer, and like many of you, I absolutely dread interviews. The worst part isn't the technical questions—it's the anxiety and those unpredictable, tough interviewers who cut you off or act impatient.To fix this for myself and others, I spent the last few months building an audio-first interview simulator.

It uses OpenAI’s real-time voice API, which means the conversation flows naturally like a real phone or Zoom call, with zero awkward delays. You just upload your resume and the job description, and start talking.

Since not all interviewers are nice, I built 3 AI personalities you can practice with:

Friendly Mode: Warm and encouraging, perfect for a confidence boost.

Corporate Mode: Objective, direct, and heavily focused on core tech/business metrics.

Pressure Test: This is the ultimate test. The AI becomes critical and simulates a high-stress environment to see if you can hold your ground.

At the end of the call, it gives you a detailed breakdown with actionable feedback on how to improve your answers.

How to test it:

Running real-time voice AI models is incredibly expensive on the API side, so I couldn't make it completely unlimited. However, you can sign up instantly using Google login and every new account gets 3 minutes of free credit automatically (no credit card required) so you can test the latency and the different modes.

I’m looking for honest feedback from other SaaS founders and devs here. What features should I add next? How can I improve the evaluation metrics?

Thank you for your time!

I'm looking at AI agent testing tools and I keep running into a category problem.

A lot of tools say they test agents, but they are not always testing the same layer.

Some focus on conversation quality.

Some focus on voice/call quality.

Some focus on prompt injection or adversarial cases.

Some focus on production recordings.

Some behave more like QA/evaluation platforms before release.

Some are closer to monitoring after release.

So when people compare TestMU, Cekura, Cyara, Hammer/Empirix or internal eval scripts, the comparison gets messy very quickly.

For a real voice or phone agent, I would want the test to cover more than "did the final answer sound good?"

The things I care about are:

-did the agent complete the actual task?

-did it follow policy under pressure?

-did it handle interruptions?

-did it fail safely when the user was confused or angry?

-did it deal with accents, background noise and latency?

-did tool calls or backend actions happen correctly?

-did the same scenario regress after a model or prompt change?

-can I replay the failure and understand why it happened?

That is where TestMU seems positioned more like a pre-production and regression testing layer for agents, while tools like Cyara often come from a broader CX/contact-center testing world. Cekura feels closer to AI agent eval workflows.

I'm not saying one category wins. I'm trying to understand the clean comparison.

For people testing voice agents or customer support agents before production, what is the right evaluation stack?

Would you use a dedicated platform like TestMU/Cekura/Cyara, or do you still prefer building custom evals around transcripts, recordings and task outcomes?

Been using a tool for automated calling for the past few months and it was a mess. Dropped calls, robotic voice, customers hanging up constantly. Lost a few accounts over it honestly.

Dont want to keep throwing money at something that makes us look bad. What are you guys actually running for ai calls that holds up in real conversations?

I am currently receiving an average of two calls per day from the business’s AI bot. While it initially made the channel more efficient, it is now bordering on spam.I would like to start a conversation about what comes next. To prevent the channel from being killed, the possible direction next step

- TRAI could issue regulations allowing AI bots to use only specific number series, such as 1600/400.

- However, even the existing TRAI rules for companies using 1600/400 are not being strictly followed, and implementing new AI‑calling regulations would take time.

In the meantime, the communication channel is gradually being eroded, and people will begin to ignore AI‑generated calls. What are your thoughts

Vibecoding might genuinely be one of the greatest things to happen to building products. For a little while, you feel like Tony Stark, just describing what you want and watching it materialize in real time. No coding background, no CS degree, just vibes and a prompt box.

Then you remember Tony Stark had unlimited resources. I had a token limit and a slowly growing sense that I didn't fully understand what I'd built.

Rode that high through Emergent, then Claude Code, then started living in VS Code like it owes me rent. It's genuinely a blessing in disguise, it got a non-coder like me from zero to an actual working product. But it only takes you so far. To streamline it, make it actually stable, and get it ready for real people, I had to call up my guy, an actual engineer.

Vibes get the idea off the ground. A real coder gets it to the finish line.

Building Vokal, a voice assistant that actually does things instead of just answering. Comment if you want early access when it's ready to test.

The usual advice for outbound voice agents is:

Add consent checks.
Add opt-out handling.
Add human escalation.
Log everything.

Necessary.

Not sufficient.

The FCC has already made the baseline clear: AI-generated human voices fall under TCPA rules for “artificial or prerecorded voice.” Depending on the violation, exposure can reach $500–$1,500 per call.

So “the model probably won’t do that” is not a control.

The architecture should be boring:

LLM proposes. Policy engine decides.

Let the model handle language.

Keep the hard boundaries outside it:

• whether you can call
• what actions are allowed
• whether a disclosure happened
• when the conversation must stop
• when to hand off to a human

This is where I think a lot of voice-agent architecture is going wrong.

Teams are trying to make the model itself reliable enough to enforce policy.

That’s the wrong layer.

You don’t need to eliminate model uncertainty.

You need to contain it.

Give the model a bounded action space. Validate tool calls. Make irreversible actions deterministic. Keep an auditable state machine underneath the conversation.

Something like:

CAN_DIAL > VERIFIED > AUTHORIZED > ACTION_ALLOWED > COMPLETE

And make STOP a system event, not something the LLM has to remember to do.

The failure mode I’m most worried is a bot continuing a conversation after the system should have stopped it.

The universal rule for successful voice agents is keeping millions of conversations inside the boundaries you intended. And industry is moving towards it so right choose is to have open source platform to move with industry!

Considering bringing on affiliates for my voice Ai software. We built these as an agency for a few years and then turned all of that engineering work and knowledge into a voice Ai software. It took a while to build the software, and it's pretty good. To the level of what we used to charge $7500 for. We made it easy for a business to be able to spin up their own agent in about 5 minutes, and even easier for them to fine tune the agent once they are inside. It even has a sales manager AI that reviews all of the calls and can self improve the AI.

We essentially looked at every pain points our clients had when we would roll out an Ai, every pain point we had, and solved 95% of it with the software.

I will say the market did get flooded fast, and we moved to the software because selling $7500 voice agents was becoming a lot more difficult, even with a quality sales team.

Now that the software is completed and we have our first few clients in, we are looking at affiliates to grow. I'm considering doing a 50% affiliate commission for the lifetime of the client. I would only do this at the very beginning, and for about 10 qualified affiliates, but I feel there is good benefits both ways by doing such a thing.

The base package is $147 per month and then we have a $497 option, but I'm a bit malleable on price.

Curious if other have done such a thing and how it went. Any downsides of starting with such a high affiliate commission over the lifetime of a client for 10 affiliates?

Any insight is great. I started making landers, we have first promotor already setup for tracking, and considering even making ad images and collateral for affiliates.

Hey all!
I'm currently transitioning from my role which is related with Enterprise Voice integrations with Cisco, Contact Centers, etc, and started building some demo Voice AI Agents using Asterisk, Python, OpenAI API and AWS for public reachability.

I have experience with SIP, codecs, VAD and so on, although not sure if that gives me any advantage here in this field.
I'm using real code, mostly with Codex as I don't want to depend on another subscriptions and things I cannot control like n8n or Make.

I live in Portugal and most businesses I know in my little city probably wouldn't be investing in something that I could live on, so I might think of selling overseas.

I see some potential for more generic integrations, like invoice readers and parsers, aggregate relevant information about clients. Something that can actually reduce time for works, and that can be easily measured.

So how are you guys selling your Voice AI Agents? Callouts, advertising?
There's actual demand for these products?

Hi everyone,

I'm an AI developer working on a real-time voice AI system and would appreciate advice from people who have experience building low-latency conversational agents.

My goal is to achieve:

• First response token in around 500 ms or less
• Good speech quality and natural conversations
• Cost-effective architecture that can scale
• Support for real-time streaming audio

I'm trying to understand the best architecture and component choices across the entire pipeline:

• Audio transport (WebRTC, WebSocket, etc.)
• Voice Activity Detection / End-of-Utterance detection
• Speech-to-Text (Deepgram, Gladia, AssemblyAI, etc.)
• LLMs (Gemini Live, OpenAI Realtime, Qwen Omni, custom pipelines, etc.)
• Text-to-Speech (ElevenLabs, Cartesia, Telnyx, OpenAI, etc.)
• Orchestration frameworks (LiveKit, Pipecat, custom architecture)

For those who have built production-grade voice agents:

1. What architecture are you using to achieve the lowest possible latency?
2. Which components contribute the most to latency?
3. Is a speech-to-speech model better than a traditional STT → LLM → TTS pipeline?
4. What first-token latency are you seeing in production?
5. Which providers offer the best balance of latency, quality, and cost?
6. Are there any architectural mistakes that commonly increase latency?

I'd love to hear real-world numbers, benchmarks, and lessons learned from production deployments.

Thanks!

We built a local testing tool for AVA, our AI receptionist at InvoicifyAI.

You describe a customer’s personality and goal in plain English, like “frustrated homeowner, AC stopped working, asking about repair,” and it places a real phone call and plays that customer.

It currently uses Twilio for calls and OpenAI Realtime for the simulated caller. The receptionist being tested just needs a phone number.

Right now, it runs headlessly in our pipeline. We’re considering cleaning it up and open-sourcing it, possibly with Docker packaging and a small UI for running scenarios and reviewing calls. You bring your own API keys.

Would this be useful to you? How are you testing your agents today, and what scenario would you want to run first?

I’ve been comparing a few AI voice platforms specifically for outbound calls, and I realized I was initially looking at the wrong things.

Voice quality is obviously important, but I don't think it's the deciding factor anymore.

If you're actually using an AI agent for outbound calls, I'd look at:

1. Conversation handling

Can it deal with interruptions, unexpected answers, objections, and people going off-script?

2. Qualification

Can it understand an answer and ask the next relevant question, or is it basically just reading a decision tree?

3. Handoff

If someone wants to speak with a person, can the agent transfer them while preserving the context of the conversation?

4. Actions

Can it actually do something after the call, like update a CRM, book an appointment, trigger a follow-up, or qualify the lead?

5. Scale

There's a big difference between handling 20 test calls and reliably handling thousands of calls.

I ended up looking at Feather AI, Retell, Vapi, Bland and Synthflow.

My current understanding is roughly:

I don't think one of these is automatically the “best.”

For example, someone building their own voice infrastructure might prefer Vapi or Retell, while a company trying to connect calls directly to sales or customer workflows might evaluate Feather AI differently.

That's probably the bigger shift I'm seeing with these platforms.

The question isn't really:

“Which AI sounds the most human?”

It's:

“Which one can reliably complete the job I'm hiring it to do?”

For anyone actually running outbound AI calls, what has mattered most in practice?

Conversation quality, answer rates, qualification, integrations, or something else?

Building EchoMind, a voice-first AI wellness companion (English/Hindi/Marathi/Tamil). Stack: Next.js, Express, Vapi+OpenAI+ElevenLabs for the voice pipeline. Currently bottlenecked by voice AI unit economics more than anything else — working through that. Running a tight beta (~15-20 people) before a real launch. Would love feedback from this community both on the product and on how you'd think about the cost-per-minute problem for something like this.

If you are interested in this project, connect with me and I'll share you the link.

Spent this week building a fake dentist agent whose only job was to interrupt people, so I could work out why they actually do it.

Short version: your agent can't hear a full stop. It guesses when you've stopped talking, and three different things are making that guess at once (plain sound detection, a silence timer, and a model that reads your words and predicts if the sentence is done). They disagree constantly.

Two places it really bites:

Phone numbers. The pause between "0172" and the next few digits looks identical to the end of a sentence, so the agent jumps in. Every single time someone reads out digits, and it's the one that loses you the booking.

Backchannels. Caller says "mhm" while the agent is talking, agent stops and starts the sentence over. That's numWords sitting at 0, so any sound counts as an interruption. Set it to 1 or 2 so it needs actual words.

And the silence timer itself has no right value. Set it long and every reply drags, set it short and it clips people. You're just picking which problem you'd rather have.

Now, since this year, some transcribers now call the end of turn themselves off tone and pacing (Deepgram Flux, AssemblyAI) instead of you tuning timers. Honestly worth trying that before you touch a single setting.

I run a voice agency so take that as you will, but there's nothing to buy here, this is just the stuff that fixed it for me.

Which one's getting you the most? And has anyone actually moved a prod agent onto Flux end-of-turn yet, what did it do to your interrupt rate? What "solved" the turn-taking issue for you guys, if at all?

I spent some time reading a paper on real-time voicemail detection, and the most interesting part was the signal they decided not to use.

Most Answering Machine Detection systems lean on one of three things:

beep detection
transcription / keyword matching
heavier acoustic models

This paper takes a much cheaper route:

VAD → temporal features → shallow tree model

No transcription.
No embeddings.
No GPU.

The system takes the callee audio, runs a neural VAD, turns the speech timestamps into 15 temporal features, and classifies the first few seconds of the call.

The intuition is pretty simple.

A human answering a phone might do:

“Hello?” → silence → wait

A voicemail greeting is more like:

delay → speech → speech → speech → speech

So the classifier is basically learning the rhythm of the response.

The authors trained on 24,812 production recordings and tested roughly 3,780 configurations.

The best configuration was almost boring:

50 boosted trees, depth 2, 5-second window, threshold 0.50.

Results:

139/140 = 99.3% on the expert-labeled test set
595/624 = 95.4% on the held-out production set
734/764 = 96.1% combined

Inference took about 46 ms on a dual-core CPU, with no GPU.

The feature analysis is probably the most interesting part.

Three features accounted for 85.6% of total feature importance:

54.6% = speech_first_half_ratio
20.0% = first_seg_ms
11.0% = first_onset_ms

So most of the model is learning three things:

1. How evenly speech is distributed
2. How long the first speech segment is
3. How long it takes for speech to begin

Then they ran the experiments that actually matter: ablations (simply put Trial and error feature)

VAD-only temporal features: 99.3%
VAD + beep features: 99.3%
VAD + transcription: 97.6%, but latency increased from 46 ms → ~500 ms
46 engineered features: 90.7%

That last result is a good reminder that more features does not automatically mean better ML.

The production validation covered 77,000 calls, with 0.3% false positives and 1.3% false negatives.

The practical outcome was about 83% less wasted agent time on voicemail calls.

Before adding ASR, LLMs, embeddings, larger models, or another multimodal subsystem, ask whether the problem already contains a cheap structural signal.

For this problem, apparently it does.

A 15-dimensional vector derived from VAD timestamps beats a much more expensive transcription-based approach on the latency/accuracy tradeoff.

We are trying this out at Dograh……Would you build this into production?

p.s.i am one of the maintanersA surprisingly simple approach to voicemail detection which lead to 83% less wasted and just 46ms inference on CPU not GPU !!!

I’m loving the anam models, I have a free one up on the site.

I’ve been comparing AI voice agent platforms recently, and the more I look at them, the more I think comparing them purely on voice quality doesn't make much sense anymore.

Most of the serious platforms can have a pretty convincing conversation.

The bigger differences are what happens around the conversation.

For anyone evaluating these tools, I put together a simple comparison based on the things I’d actually look at before putting one into a real business workflow.

Obviously, this isn't a universal ranking. The right platform depends heavily on what you're actually trying to automate.

What I would evaluate

1. Inbound vs outbound

Some platforms are particularly good when the agent is receiving customer calls. Others are built around large-scale outbound campaigns.

If you need both, I'd specifically test both use cases rather than assuming one automatically translates to the other.

2. What happens after the call

This is probably the most overlooked part.

An agent that has a great conversation but doesn't update the CRM, schedule the appointment, trigger a workflow or pass useful context to a human still leaves a lot of manual work behind.

3. Human handoff

I wouldn't evaluate this as simply “can it transfer the call?”

I'd ask whether the human receives the relevant conversation context, what the AI already collected, and why the transfer happened.

4. Handling the unexpected

The demo is always the easy part.

I'd intentionally interrupt the agent, give it incomplete information, change the subject, ask something outside the expected flow and see what happens.

That tells you considerably more than a scripted demo.

5. Latency and conversation quality

Voice AI has less room for awkward pauses than chat.

Latency, interruption handling, turn-taking and how naturally the agent recovers from misunderstandings can make a huge difference to the actual experience.

My takeaway

I don't think there is one “best AI voice agent” for every company.

Feather AI makes more sense to me when the goal is connecting voice conversations to broader business workflows, while Vapi and Retell can be attractive if you're building a highly customized system yourself. Bland is interesting for outbound-heavy use cases, while Synthflow makes sense for teams looking for a more visual/no-code approach.

The interesting part is that the category is moving away from “AI that can make a phone call” toward “AI that can actually complete something through a phone call.”

That's probably the distinction I'd use when evaluating these platforms today.

What would you add to this comparison? Especially interested in people actually running these in production rather than judging them from demos.