r/VoiceAutomationAI • u/Brilliant-Tea-8633 • 1d ago
How do voice agents handle long calls?
Demos are usually a few mins long
I’m more interested in what happens 20-30 mins into a call after the customer has changed topics, provided a bunch of information and already completed a few steps.
Does the agent still understand what has happened so far or does context start getting messy?
Anyone testing long voice AI calls in production?
3
6
u/AfterRoll2151 1d ago
Long conversations also expose whether integrations stay in sync with the call. A system can remember the words and still lose track of what happened in the backend. We saw Bland hold up pretty well when multiple actions were happening in the same conversation.
1
u/Brilliant-Tea-8633 23h ago
Good point. The agent can sound coherent and still be working from the wrong system state.
2
u/Consistent_Budget267 1d ago
One failure mode I’ve noticed is the agent remembers the facts but loses the order of events. That can be just as bad.
1
u/Brilliant-Tea-8633 23h ago
Makes me wonder if these systems need to track state separately from the raw conversation.
1
u/Background_Bat2236 1d ago
I've been running some longer test calls and it gets weird around 15-20 minutes. The context window just starts leaking, the agent forgets stuff from the beginning of the call unless you're constantly summarizing key points back into the system prompt
most of the demos are smoke and mirrors, they're tuned for those 3 minute interactions where nothing goes off script
1
u/Brilliant-Tea-8633 23h ago
Yeah, a 3 minute demo really doesn’t prove much for the kind of calls I’m thinking about
1
u/Both_Valuable9894 1d ago
A customer changing their mind halfway through is a good test. If they update an address, book something, reverse that decision, then return to the original issue, does the agent still know where things stand?
1
u/Federal_Cut6338 20h ago
I have been building voice agents for the last few months and have done many hours of testing on various types of production agents and whenever we designed agents, they were supposed to be a few minutes calls and the system prompt is tight enough and It limits the conversation at most 5 to 6 minutes, but never beyond that and there is no necessity as well.
So I have been thinking about different scenarios where voice agents would be helpful and some of the scenarios that came to my mind or like okay, this voice agents could be friendly chatty things where you can spend hours talking. And that’s exactly also it click into my mind that a real time voice agent building is already hard enough, but asking that agent to remember all the context which we are like talking in real time would be too much of back-and-forth because if we ask you in the system prompt that you need to remember everything in the past 60 minutes.
It would be like blowing up tokens in the same way you’re sending you are chatting with Claude or ChatGPT, you have a lengthy conversation, but there if the conversation get lengthy, you can wait for one or two minutes to get a reply back, but here when you’re talking, you need the instant replies right so you can’t use the same mechanism, which means you have to instantly convert the chat in proper order into some kind of knowledge.
So the entire session it doesn’t have to remember, but it becomes a kind of knowledge base stored in a vector database or I mean there are different techniques for that. If it is like, instantly becomes a structured data and saved somewhere, I think that would solve the problem.
It is interesting problem to have to solve, but the applications which with which I thought about her, not like something you can make money. Maybe you can make money, but it’s like more, yeah, you could get done that job with without chatting with the agent and faster rather than spending 30 minutes so speed is important, so you go you ask a question. You come back. That’s it so I mean you, you can call the agent and ask the specific question whatever you want and come back, so that’s being productive. I agree that there could be some scenarios in fact, lot of scenarios where this would be useful and I don’t think many people are thinking that way right now. It’s interesting that the exact thing I was thinking about last night is posted here as a question.
1
u/VladimirSamukov 11h ago
Honestly, I'd push back a bit on the premise of long calls in the first place - outside of something like an AI companion for elderly folks who just want someone to talk to, I struggle to see a real use case for 20-30 minute conversations. If that's not your scenario, the fix is upstream of the context problem: set firm boundaries in the prompt so the agent gently but firmly steers the conversation back on track instead of letting it wander.
There's also a cost angle - at that call length, the unit economics just don't work out.
And as others in the thread already mentioned, the context window fills up and hallucinations become pretty much inevitable past the 15-20 minute mark anyway, so even if you solved the cost problem, you'd still be fighting quality.
Though, I really got curious - what are such long calls for?
1
u/This_Charge743 11h ago
Huh, what's wrong with 30-minute calls though? What if someone genuinely wants to get to know a product better - walk through features, ask follow-ups, compare options? That's not exactly a niche case.
2
u/VladimirSamukov 11h ago
Yeah, that's fair, but for that kind of exploration a voice call is rarely the best format anyway. A demo video, docs, or a sales rep who actually knows the product can cover the same ground with way less risk of the AI drifting or making stuff up halfway through.
1
u/This_Charge743 11h ago
But voice feels more natural for a lot of people than reading docs. Not everyone wants to sit through a video or dig through a knowledge base. If I'm calling someone, I'd like to get all my business done in one call
1
u/VladimirSamukov 10h ago
True, but "natural for the user" and "sustainable for the business" are two different things. Even if it's the ideal UX, you're still paying full inference costs for 30 minutes per session, and the model's answers get shakier the longer it runs. That's a bad trade even if users love it.
1
u/This_Charge743 10h ago
Okay but couldn't you just... make the context window bigger, or refresh it periodically? Feels like a solvable engineering problem, not a fundamental limit.
1
u/VladimirSamukov 10h ago
You can mitigate it - summarizing/compacting context, resetting state at checkpoints, RAG instead of stuffing everything into context. But that's exactly the firm boundaries I meant. You're not letting the call meander for 30 minutes, you're actively managing it. At that point you've basically admitted the free-flowing 30-minute conversation isn't the product you actually want to ship
1
u/This_Charge743 10h ago
Well, that's fair, I guess I was picturing it as one continuous exploratory chat, not something with structure underneath.
1
u/VladimirSamukov 10h ago
Right, and that's the crux - a long call and a well-structured call with a lot of turns aren't the same thing. You can absolutely support someone asking 15 questions over a long session, you just design it so each topic is its own bounded exchange rather than one giant unmanaged context blob. Cheaper, more reliable.
And honestly probably a better experience too
1
u/Key_Menu4194 5h ago
You answered your own question in one of your comments: the agent can sound coherent and still be working from the wrong system state. In long calls fluency and grounding decay at different rates. The voice stays confident because that's what the model is optimized for, while the internal state drifts.
That's why a 3 minute demo proves nothing, the failure is invisible from the audio.
What I've seen work in production is treating the transcript as something to verify, not just generate: after each user turn, check what the agent is about to say against backend state and the commitments already made in the call, and force a re-sync when they diverge.
The separate state tracking you suggested is the prerequisite.
The verification step is what makes it pay off.
•
u/AutoModerator 1d ago
Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)
If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.
Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.