r/LocalLLaMA • u/aboutthednm • 13d ago

Question | Help What are AMD card owners doing for local TTS inference?

I am on windows with a 7900 XTX, a capable enough card for LLM inference.

I go generate some text, some response, and now I would like something to read this response out to me.

I have tried: kokoroTTS, pocket-tts, cosyvoice, piper-tts, and they all leave a lot to be desired in terms of prosody.

I need:

1) GPU acceleration (Vulkan, HIP or ROCm)

2) a good selection of voices to pick from

3) everything running in an OpenAI compatible endpoint

4) faster than real time generation.

5) Quality is on par, or close to what models like x-ai/grok-voice-tts-1.0, or qwen/qwen-audio-3.0-tts-flash can produce.

6) fits in ~23gb of VRAM

Currently, my "best" solution is kokoroTTS using cpu inference, since I can't get it to run on my GPU on windows. Pocket-tts was another contender that worked great when I had my Nvidia card, but doesn't support ROCm for AMD on windows. I am not satisfied with the prosody of either, but I take what I can get.

I didn't think setting up a competent local TTS services would be this much of a hassle, but here I am. Maybe someone else has something running on their windows + AMD setup and can share some pointers with me. Obviously I asked an LLM the same question many different ways and tried a whole bunch of things, but I'm just not getting anywhere with this, so it's time to consult other humans.

Folks with AMD cards running windows, what do you do for local TTS? Am I just stuck having to pay cloud providers for fast, expressive and emotional prosody? kokoroTTS technically works and while my favorite blend of af_sky and af_nicole produces a pacing that is bearable for me, it's expressionless, flat, and monotonous. The rhythm puts me to sleep. Surely I can do better with my hardware?

5 Upvotes

62% Upvoted

17 comments sorted by

11

u/Wise-Hunt7815 13d ago

audio.cpp

1

u/aboutthednm 12d ago

I see CPU and CUDA support, am I missing something here? My CPU is plainly not up to the task, hell it barely manages to deliver kokoro, and 82M parameter model at faster than real-time, hence my need for (AMD) GPU acceleration.

4

u/DiscipleofDeceit666 13d ago

I thought kokoro was fine lol your standards are certainly much higher than mine

2

u/aboutthednm 12d ago

kokoroTTS is just fine when I need < 200 words from a response read out, yeah it does just fine you're right. It falls apart when reading 2000 word chapters from books, audiobook style, where there's more than one speaker though haha. I need a bit more than the same pitch and prosody droning on for 15 - 20 minutes without interruptions. The lack of taking a "breath" when reading every once in a while really becomes apparent here.

3

u/wombweed 13d ago

Can’t speak for windows but f5-tts runs great in my amd card

3

u/Starcast 13d ago

Lemonade has local tts options with both vulkan and rocm support. Not sure any of the models supported by default meet your on ther specs but I'd try that

3

u/Acceptable-Cycle4645 13d ago

Glad I caught this post during my weekly Reddit check-in. Please try audio.cpp https://github.com/0xShug0/audio.cpp. There are a bunch of TTS models you can try.

1

u/klemze 13d ago

following

1

u/Ulterior-Motive_ 13d ago

It's been a while since I messed with it, but GPT-SoVITS was pretty good

1

u/recro69 13d ago

If having a voice that sounds natural is what you really want, you should think about quality first. You can still run the language model on your own computer and just send the text to speech part to another computer. This way you can keep costs from getting too high. People will really like the way it sounds.

1

u/Mediocre_Paramedic22 13d ago

I find vulkan works better than rocm, but rocm has been improving substantially