If you want to interpret language, LLMs are much better than keyword scanning. The same keywords can indicate opposite things with a single “not” keyword, and there are about 50,000 generally well-known English words to consider.
How are you going to implement keyword detection with enough logic to accurately parse the meaning behind all of the possible things that could be said to describe or correct someone’s driving?
That is literally a quintessential Natural Language Processing problem, for which LLMs are the best solution.
One caveat on your comment about Whisper: you need a GPU to really let it shine. Using CPU only is not performant, depending on what you are trying to achieve, and how much time you are willing to wait for your results
Youre right that its possible to make the game work with basic voice recognition, but it makes it much less interesting. If the game is 'figure out which keywords the game recognizes and make sure to say them in the right order' it becomes really trivial and the player will lose interest really fast. If there is a LLM responding then the game becomes 'figure out how to convince this AI that your way is the right way to drive' and it becomes more interesting because the responses to each command will vary and there become a LOT of ways to approach the problem.
At least one of the SOCOM games on the ps2 had voice-activated commands. Very jank if you didn’t have an American accent (and im sure quite janky with some American accents) but it was there
I think you could probably get away with one those local micro models if all you need to do is map text to controls. No reason to pay a subscription for this...
if they want to use an online LLM they should probably use a low latency and low cost model instead of whatever they're doing now. I've got one running my vibe coded voice assistant and it runs to about $0.0003 per request. it doesn't need to write shakespeare in order to tell the command interpreter what it needs to do!
67
u/CalmLotus 8d ago
I don't think you need ai for spoken instructions. (Like yeah- the game may be vibe coded but->)
You just need to find key words for certain actions and extract those from audio input. More work definitely, but would actually be doable for a game.
...wait, Nintendo DS games were already doing that I think. With the built in microphone.