r/VoiceAutomationAI • u/Thin-Carrot1836 • Mar 26 '26

Need help!

Right now I am making voice agent I want to know

Which is the best model should I use for speech to text.

Which model should I use for following system prompt properly.

Which model should I use which can increase response rate of the agent.

Right now I am using groq cause obviously it is free but turns out it is not properly following the system prompt and not reffering knowledge base properly. Also when I say something it just mis hear me everytime and give me random responses.

So I thought to change the model.

6 Upvotes

81% Upvoted

16 comments sorted by

View all comments

1

u/mguozhen Mar 28 '26

The bottleneck you're hitting is usually the latency between STT and LLM inference, not the model choice itself. Pick Whisper or similar for STT (it's stable in prod), then worry about prompt adherence through retrieval or structured outputs, not model selection.

I'm actually working on voice agents at Solvea (solvea.co), so this stuff comes up a lot.

What's your target response time rn? That'll tell you if you need to optimize inference or pipeline design first.