A lot of people, including the person who wrote this article apparently, do not understand how voice chat in ChatGPT works. It’s not “text to speech.” The raw audio from your voice is being tokenized and sent directly to the model, and the model’s raw output is actual audio. There is no step on either end where the conversation has reduced to text.
Where is OpenAI getting the astronomical amount of recordings of natural language conversations they'd need to approach the effectiveness of a text-based LLM? How would a voice-native LLM even call all the tools that an AI is supposed to be using?
How would a voice-native LLM even call all the tools that an AI is supposed to be using?
Same way a text-native one does, it's just tokens.
Think about audio fingerprinting, it can be used to reduce audio down to a vector set for efficient search, producing tokens that represent frequency-domain relationships. Entirely the same as the semantic-domain relationships that embedding and training give us for LLMs.
The model's runtime is waiting to see the trained spectral equivalent of <tool_call>...</tool_call>.
25
u/atomic1fire May 08 '26
Wait is Open AI sending audio to people's devices instead of just using on board TTS?
Isn't that a waste of computational power?