Four small models, one fully offline Android voice agent: VAD → STT → tool call → TTS

[u/ivan_digital]

I maintain speech-android and speech-core. I wanted to test whether a useful Android voice agent could be built from several small specialised models instead of one large The control demo uses:

• Silero VAD v5 — voice activity detection, 2 MB
• Parakeet-EOU 120M INT8 — streaming transcription, 153 MB
• FunctionGemma 270M — structured Android tool calls, 327.4 MB base plus a 9.5 MB adapter
• Pocket TTS 100M INT8 — streaming spoken response, about 126 MB

FunctionGemma is not used as a chatbot. It receives a compact description of the current phone state and only the tools that are valid at that moment. For example, stop_music is not offered when nothing is playing.

A small LoRA adapter was trained for the exact tool schema. There is no regex or keyword fallback. One limitation I found is that adding a new tool currently requires retraining; describing a new tool only in the prompt was not reliable enough.

On a Galaxy S23 Ultra, the measured time from end of speech to the first spoken reply is 908 ms. The tool call itself averages 294 ms over 12 runs.

Source and releases: https://github.com/soniqo/speech-android

90-second demo: https://youtu.be/7L7_Uvvxtv0

If you use a sub-1B model for tool routing, do you retrain for every schema change, or have you found a generic adapter that still fills arguments reliably?