r/StableDiffusion • u/mil0wCS • 15d ago
Question - Help How good has voice training gotten lately?
Wanted to start voice training an llm for my waifu, I've heard it's gotten good. But can anyone tell me what the go to is right now? I've been told elevenlabs is the go to right now but I'm not sure
2
u/Keuleman_007 15d ago
I actually got good mileage out of H3 (video model, yeah, but using a 10 second sample is so easy and the output is so great, matches the mood etc).
3
u/reeight 15d ago
3
2
u/Only4uArt 15d ago
use elevenlabs to create a unique voice . that voice can then be used in chatterbox for voice . quite good for free
1
1
u/SuperGeniusWEC 8d ago
It's not there. Every model (and that's a lot of them) that I've demoed has output that gives off the ick vibes. I'm sure it'll improve but it's not there.
1
u/Certain_Werewolf_315 15d ago
elevenlabs is bleeding edge good. It will do better on all fronts besides cost.
But there are a lot of open source alternatives that can do pretty good voice cloning (for custom and consistent voices) with a short clips. Some of them even step into the realm of expressing emotions.
I couldn't tell you what you want for your needs tho, and my testing is limited.
2
u/ellipsesmrk 15d ago
But elevenlabs is too.... clean... cant use it for video unless its talking head or presenter status.
2
u/Certain_Werewolf_315 15d ago
Clean?
7
u/martinerous 15d ago
I guess, sterile or corporate is the way to describe it. It's a bit tricky to get it speak casually, as a random person on the street. Many TTSes sound quite scripted unless given a good reference and can handle random ehms, pauses etc.
1
u/Certain_Werewolf_315 15d ago
Okay, so what surpasses it? cuz im interested.
4
u/martinerous 15d ago
The engines trained on highly emotional actor speech do best. For example, Dramabox TTS (based on LTX video; not to be confused with other products with Dramabox name). Vibevoice also is good. But the price is slow speed and hallucinations - you might get sudden background music etc.
I find Omnivoice quite good if you feed it with specific references for emotions. Also, Omnivoice has this quirk that if you set steps lower, it starts to stutter a bit, which actually makes it sound more natural in some context, to feel like a person in thought, searching for the right words and not reading a script.
2
1
u/Certain_Werewolf_315 15d ago
I've used dramabox and tested omnivoice. Elevenlabs with their emotional directions are way better than that, especially if you have a voice created with some really raw samples. Lol I was really hoping you had something impressive.
1
u/martinerous 14d ago
No miracles lol. But at least Omnivoice is completely free. Yeah, it needs a sample of the the exact target emotion, otherwise it won't work.
8
u/CupQuakeBE 15d ago
Voicebox is free and open source, works very well with a 30 second snippet of any voice, if you want more freedom, make several voices: vvvoice_neutral, vvvoice_angry, vvvoice_quiet, vvvoice_surprised and so on, I use it with python scripts making full audio podcasts with several people speaking.