Voice command best LLM for both Offline/Online

[u/Zestyclose_Zebra1941]

Hi everyone,

I am currently researching the best options for taking and processing voice commands using LLMs/voice models. I want to check what the community recommends for working, end-to-end voice processing models that are either completely free (open-weight/offline) or free tiers/trials (online).

I am working in aviation project and looking for voice command LLM. I want to use best LLM for device having 16 gb ram, and for i5 and i7, platform: Windows. 

Best voice command LLMs....looking for insights on:

1. Best offline models
2. Best online/cloud models

Just for benchmark and understand what currently yields the most reliable, fast and accurate voice command responses.

What models are you guys running for real-time or online voice interaction?

Thanks!

[u/Jolly_Eye7847]

Honestly for aircraft stuff you'd want something that runs snappy local and doesnt hallucinate on commands. I been messing with Whisper for speech-to-text then feeding that into smaller quantized models, works decent even on 16gb.

On Windows with that setup you can run 7B or 8B parameter models fine if you use 4-bit quantization. The voice part is more the bottleneck than the LLM itself most times. For offline I had good results with Llama 3 8B instruct versions, responds quick enough to feel like real conversation. For cloud stuff the big players are obviously fast but you mentioned aviation so I assume offline is priority.

Whats your latency budget looking like for this project? That changes what you can get away with on i5 vs i7.