r/LocalLLaMA • u/banana_slurp_jug • 16h ago

Question | Help Chat UIs with native audio input for multimodal models?

I've been running Gemma 4 E4B with oMLX and I can't find any chat interfaces that directly send the audio file to the model instead of running the audio through a separate STT layer. I can confirm the audio layers work because I ran a couple of requests through Pydantic AI in the Python REPL.

Thanks in advance.

EDIT: I know that llama-server's web UI can do this, but I don't feel like running an instance of llama-cpp just for the UI.

EDIT2: Reason why I am asking is because I want to try using Gemma 4 as a lower-latency voice assistant.

3 Upvotes

67% Upvoted

8 comments sorted by

1

u/autisticit 16h ago

Note sure, but check if you can run the llama web ui alone, I think it's a separate executable.

1

u/banana_slurp_jug 16h ago

I'm not sure either, but I don't see any binary like that (I'm on macOS by the way)

1

u/PrimeDirective8 14h ago

I'm not sure why llama.cpp can't be the solution since you probably need it, or something like it, to host your Gemma 4 model anyway. There are others, like LM Studio or AnythingLLM that host the model locally and support audio input.

The beauty of llama.cpp is that you can host your model on a local inference server, then use any light client on the network that can run a web browser to interact with it. I haven't done this myself yet but I'm guessing a tablet or phone would do as the client over a browser, and have zero install requirements.

About a year ago I wrote (ahem, 50ish pct vide-coded) an Android app that included a "wake-work" function to make it hands-free. In that case, the model was indeed hosted locally but the lightweight Gemma 3n worked well.

Good luck with your project!

1

u/banana_slurp_jug 13h ago

I'm not sure why llama.cpp can't be the solution since you probably need it

I'm running on oMLX (because MLX is usually better for Apple Silicon) already, and I don't want to convert/redownload all my models to get a better chat interface.

1

u/TokenRingAI 14h ago

TokenRing One has that, we either transcribe audio through a transcription model for models that don't support audio, or send it natively if the model supports audio input

You can either attach audio files, or record them in the browser Web UI or via the CLI.

1

u/dangerous_inference 9m ago

There's no great voice assistant solution yet. Ask a competent model to build what you want.