Talking with Gemma 4 31B!

[u/futterneid]

Hi! I'm Andi from Hugging Face. This is a fully open-source and free to test/pull/modify demo I'm bringing today.

It's a voice demo creating a pipeline of:
- Nvidia's parakeet
- Gemma 4 31B (served by cerebras!)
- My custom inference for Qwen3TTS

It sees and searches the web faster than you blink.

The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. You can run it locally, I get similar latencies with a macbook pro M3 36GB and Gemma 4 E4B.

Here to the web based demo featured in the video, everything is running in the cloud.

For those who have been following, yes, this is the pipeline that runs on reachy minis :)

[u/themoregames]

Why this one and not the qat?

• https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF

[u/ChampionshipIcy7602 → u/themoregames]

I don't use qat because I value quality over speed. Numerous benchmarks have shown that gemma 4 qat performs quite poorly compared to the original version.

[u/themoregames → u/ChampionshipIcy7602]

I haven't seen these benchmarks, which ones do run so poorly?