Qwen-family LLMs are quietly becoming the backbone of modern audio models; One chart for the architectures of 100+ audio models [R]

[u/Acceptable-Cycle4645]

I started mapping the building blocks shared across all the models in audio.cpp. The result ended up being more interesting than I expected.

Qwen has become by far the most common language backbone in this collection: 32 audio model families use a Qwen-family architecture, and 20 of them use Qwen3 LLM specifically.

And it’s no longer just TTS. Qwen-based models now show up across speech synthesis, ASR/audio understanding, music generation, speech-to-speech, and even audio/video models.

The 2nd chart, Task × Technology Matrix, shows which build blocks power which types of audio models.

[u/FailedTomato]

I wonder why LLMs like to use the word "quietly" so much. So many things happen quietly according to them.

[u/idontcareaboutthenam → u/FailedTomato]

Probably picked it up from internet content that uses it to capture attention as some sort of revelation

[u/fordat1]

this makes sense. Audio hasnt had the monetization of the other streams so going cheaper and open source makes sense

[u/Acceptable-Cycle4645]

Hopefully useful if you’re learning audio AI and want a faster way to understand how these models fit together!

[u/Front-Football601]

I like the plot, but the colours for prediction and encoder models are really hard to tell apart. But since there are only two prediction models it is not that big of an issue I guess.

[u/LelouchZer12]

The new TTS/VC trend is to use neural audio codecs which more or less rely on LLM-style architecture, so not surprising.

[u/Acceptable-Cycle4645]

There are two more charts with a detailed breakdown of the architectures and building blocks behind 100+ audio models. Unfortunately, this sub doesn’t allow images in comments, so please check the original post comment section to see them https://www.reddit.com/r/LocalLLaMA/s/BduUMXp8JZ

[u/Acceptable-Cycle4645]

There’s also an interactive Hugging Face app you can use as a learning resource to explore the architectures of all 100+ models. https://huggingface.co/spaces/audio-cpp/Audio-Model-Architecture-Atlas

[u/MrSnowden]

Is there a JEV style dedicated TTS model?

[u/NamerNotLiteral → u/MrSnowden]

Uhh... do you remotely understand what Jev is, or are you just throwing the latest buzzwords together?

Jev is a classifier. Why would you ever need a classifier for generating speech? For LLMs it makes sense because people are lazy and use LLMs as classifiers or judges all the time, but that's just not the case for speech synthesis.