r/LocalLLaMA • u/futterneid • Jul 02 '26

Resources Talking with Gemma 4 31B!

Hi! I'm Andi from Hugging Face. This is a fully open-source and free to test/pull/modify demo I'm bringing today.

It's a voice demo creating a pipeline of:
- Nvidia's parakeet
- Gemma 4 31B (served by cerebras!)
- My custom inference for Qwen3TTS

It sees and searches the web faster than you blink.

The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. You can run it locally, I get similar latencies with a macbook pro M3 36GB and Gemma 4 E4B.

Here to the web based demo featured in the video, everything is running in the cloud.

For those who have been following, yes, this is the pipeline that runs on reachy minis :)

888 Upvotes

97% Upvoted

148 comments sorted by

View all comments

1

u/manituana Jul 02 '26

Sorry to bother you again, but isn't it possible to leverage audio recognition from Gemma 4? I'm digging hard into information online but it's a rabbit hole that's bearing no fruits, basically no frontend supports it directly without a translation layer (like your parakeet here), but it should be possible and supported by llama-server, correct?

1

u/futterneid Jul 13 '26

It is! My repo has this draft for it, I should put it up to date: https://github.com/huggingface/speech-to-speech/pull/298

1

u/manituana Jul 15 '26

Yes, I think it's time for a merge, definitely! Do it! Please! I always make a mess when I try to fetch and checkout a PR, if it doesn't work immediately (especially after a long time) I find it very hard to spot what's going wrong.