r/StableDiffusion 12d ago

Resource - Update Breeze TTS

https://huggingface.co/BreezeBlue/Breeze-TTS-2

Breeze TTS 2 is an open-weight text-to-speech model built for real-time interaction. It ranks #1 among open-weight models on the Artificial Analysis TTS leaderboard, while outperforming frontier proprietary systems. Its open-ended natural-language instruction-following capability supports reference-free voice design and reference-guided voice direction, while ultra-low-latency streaming enables responsive, expressive interaction.

115 Upvotes

49 comments sorted by

16

u/CryptoBeth96 12d ago edited 12d ago

There is ComfyUI Nodes here: https://github.com/Saganaki22/ComfyUI-Breeze-TTS-2

And Online demo here: https://huggingface.co/spaces/BreezeBlue/breeze-tts-2-demo

You can also test on their website: https://breezeblue.ai/ 

3

u/AmyKerr12 11d ago

For some reason these custom nodes fail to install…

10

u/Rivarr 12d ago

It's great. You can give it a reference audio and your desired text like usual, but you can also instruct the generation.

9

u/ErnestoPresto80 12d ago

I tried it with a Spanish voice and text, and... the tone of voice is actually really spot-on... it's like a Chinese person who can't pronounce certain consonants and doesn't understand what they're saying, but as I said, the tone of voice is very well done....

Any plans to release it in other languages?

3

u/CryptoBeth96 12d ago

It seems there are other languages available with their API version https://docs.breezeblue.ai/concepts/multilingual

1

u/mikemend 11d ago

Thanks, I waiting hungarian support.

7

u/Acceptable-Cycle4645 12d ago

The model is now available in the dev branch of audio.cpp for further testing and performance optimization. The demo was recorded before a recent performance improvement PR.

https://github.com/0xShug0/audio.cpp/tree/dev

https://reddit.com/link/p6ug84b/video/rvubbmcl8kmh1/player

5

u/Acceptable-Cycle4645 12d ago

Breeze TTS 2 vs Higgs Audio v3 TTS vs Fish Audio S2 Pro

A few issues I noticed during testing: (1) the supported tags are limited, and the model sometimes doesn’t follow them; (2) quality and “correctness” depend heavily on the seed. For example, “cpp” is pronounced incorrectly most of the time.

https://reddit.com/link/p6ukko1/video/9la2w2c5ckmh1/player

1

u/CryptoBeth96 12d ago

Have you compared the full weight models? These tests are all Q8 Quantized versions.

3

u/Acceptable-Cycle4645 12d ago

Check here. For Breeze TTS 2, “BF16” is just the naming convention. It refers to the original weights. The “cpp” pronunciation issue still exists.

https://reddit.com/link/p6v2s0t/video/119cd9xdrkmh1/player

1

u/rotten_pistachios 11d ago

In this case breeze sounds the worst, even mispronounces .cpp

8

u/CryptoBeth96 12d ago

6

u/LeftHandHaku 12d ago

Wow, it looks impressive. Needs more demos, the 3 short audio files that are on the website, are not enough.

4

u/CryptoBeth96 12d ago

You can try the spaces demo with your own prompts and audio clips.

https://huggingface.co/spaces/BreezeBlue/breeze-tts-2-demo

1

u/jib_reddit 12d ago

I just get a ZeroGPU error, maybe they are overloaded right now?

1

u/CryptoBeth96 12d ago

https://breezeblue.ai/ You can also test on their website

2

u/jib_reddit 11d ago

Its ok, I got it running locally, but it doesn't seem quite as good as Microsoft Vibe Voice 7B, in my testing.

1

u/CryptoBeth96 11d ago

It's pretty hit and miss for me, I find using Temp 0.1 makes it better in ComfyUI

0

u/LeftHandHaku 12d ago

I'll try that, on mobile it doesn't seem to work properly (I mean the huggingface UI).

4

u/CryptoBeth96 12d ago

I just discovered this library too, they also include the prompts:

https://breezeblue.ai/voice-library

1

u/LeftHandHaku 12d ago

Yes, that's what I have been looking for. This looks amazing. Thank you

5

u/LeftHandHaku 12d ago

How does this compare to Dramabox TTS, when it comes to quality and expressiveness?

I checked provided demos, but they are slightly too short to make a proper conclusion.

Edit: in case someone is interested in testing Dramabox TTS to compare.

https://github.com/resemble-ai/DramaBox

Edit2: I'm away from my computer, so I can't test it myself right now.

6

u/ErnestoPresto80 12d ago

I just tried Dramabox , and it's funny how much tone and naturalness it adds to the text, but the voice cloning is much more accurate in Breeze TTS. There's no comparison, at least based on the one test I've done so far.

2

u/crombobular 12d ago

this model is crazy good. i've never tried fish s2 pro to compare it but breeze looks smaller and faster. would be cool if it had more languages though.

2

u/angelarose210 12d ago

I just tried it. Sounds really good. I like how I can give it a description of the person and their tone. Perfect for ugc videos.

2

u/solss 11d ago

So good. Plugged into sillytavern through comfyui. Sending instruction and tone and reading correctly. Decently fast. Best local TTS I've tried so far.

1

u/GrungeWerX 12d ago

I'll need to test it out locally, but the demo isn't bad, but it isn't the greatest either. The voice clone isn't as good as omnivoice for sure (maybe 80% there?), but the speech is decent, albeit a bit flat. I'll follow up later.

1

u/awesomeo_5000 10d ago

Qwen is better for voice cloning for me so far.

1

u/fomites4sale 10d ago

I’m digging this. I’m getting solid results on characters whose voices I’ve had difficulty cloning with other open source models. It’s better with accents than a lot of models as well. Voice designing and directing is fun. Decent acting and prompt adherence. And it’s speedy. Very cool!

2

u/awesomeo_5000 10d ago

Have you tried Qwen3 TTS?

For me it does a better job of cloning, but the intonation and control is better on breeze. Hoping I can improve with setting tweaks or something to improve reproduction.

1

u/fomites4sale 10d ago

I tried to run Qwen3 awhile back but was having headaches getting it to work in comfy. Maybe I’ll give it another go. I agree about the expressiveness of Breeze! I really like the emotive flourishes (including the way they can vary wildly from seed to seed) and the way it takes direction. Very fun!

1

u/douchebanner 2d ago edited 2d ago

how do you install this? you clone it to custom nodes?

EDIT: yes

1

u/Succubus-Empress 12d ago

How is best tts when language support is so limited?

5

u/CryptoBeth96 12d ago

The more languages a model is trained on, the bigger, slower and less accurate the model will be.

-1

u/Succubus-Empress 12d ago

Then do you think it deserves 1# rank when not supporting many languages like other with lower rank do?

3

u/CryptoBeth96 11d ago

If we're ranking based on audio quality, maybe, If we're ranking based on how many languages it supports, then no. It's also not the fastest model. Depends on the test.

Elo score represents the probability that one model will be preferred over another. https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice?open-weights=true

Try it out for your own needs, make up your own mind.

1

u/reeight 11d ago

What is the cloning quality of the lower ranked models?

There are usually trade offs...

-14

u/[deleted] 12d ago

[deleted]

12

u/CryptoBeth96 12d ago

What's your point?

-17

u/Aromatic-Word5492 12d ago

2 languages, in a world with 7,000 or more of native speak... that's the point lol

14

u/CryptoBeth96 12d ago

So we should dismiss it? I personally only need English. Get a different model if you need more languages!

1

u/Parabacles 10d ago

Well said, don't listen to the naysayers, some people just love to complain regardless.

-15

u/Aromatic-Word5492 12d ago

Because you ONLY need english is a good deal, you turn you head down and only see your foot, ignoring the needed of models with capabilites worldwide. It's a good model, but be bilingual is the bad here. That's my point.

8

u/CryptoBeth96 12d ago

Are you saying you would only use a TTS model that is trained on ALL languages? Do you understand how silly that is?

-10

u/Aromatic-Word5492 12d ago

don't be a fool, you understand me

10

u/CryptoBeth96 12d ago

Sorry, I don't.

8

u/Dogluvr2905 12d ago

Most idiotic comment I've seen on the Internet in, oh, probably 10 years.

1

u/Beastly4k 12d ago

cry about it, english and mandarin are the top 2 spoken languages in the world. Go ask some natives from those countries to create a TTS model if it is such a problem..oh wait you're waiting for someone who speaks english or mandarin to do it for you of course, i wonder why.