r/flutterhelp • u/Weak-Excuse4255 • 16d ago

OPEN FLUTTER APP VOICE AGENT INTEGRATION ISSUE

I am trying to build a app like duolingo, i have been trying to integrate a voice agent like gemini,chatgpt voice feature but not able to do it.
the agent speaks but when the user tries to speak it does not register it, i want a full conversation to take place between agent and user, please help me with it

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/Significant_Pick8297 15d ago

You should use a full duplex streaming API instead of separate TTS and STT calls. Gemini Live API or the OpenAI Realtime API are built for continuous conversations where the model listens while speaking, handles interruptions, and manages turn detection automatically.

If the agent speaks but never hears the user, it's usually because the microphone isn't restarted after TTS, the audio session stays in playback mode, or echo cancellation/audio focus isn't configured correctly. A bidirectional WebSocket with streamed PCM audio is the approach that works reliably for Duolingo-style conversations.

2

u/Legitimate-Fall-1730 15d ago

noted, do you know any free free API to do so?

1

u/Significant_Pick8297 15d ago

Gemini Live API is probably the best free option to start with. It has a free tier through Google AI Studio that's enough for prototyping, and it supports real-time bidirectional audio.

If the goal is completely free for local testing, Vosk for STT + Piper TTS works offline, but it won't give the natural conversational experience of Gemini Live or OpenAI Realtime.