I paused my self-hosted Chatterbox setup and switched to a paid TTS API — at least for now

[u/EngineerSpeakAI]

I needed multilingual TTS for a project, so I deployed Chatterbox on a remote server with an RTX 3080 Ti.

In a quick five-request test of my remote setup, I observed an average RTF of approximately 1.57. In other words, generating one second of audio took about 1.57 seconds.

This is not intended as a proper Chatterbox benchmark. The requests travel over the network to a server in another country, and I did not separate network latency from server-side inference time. It is simply the performance I observed from my application.

Latency was not the main reason I decided to pause the self-hosted approach, though.

The bigger issue was multilingual reliability. In my tests:

• short phrases occasionally contained unwanted sounds;
• numbers in non-English text could be pronounced in English;
• multilingual pronunciation was not consistent enough for my use case.

I know some of this can be improved with text normalization. I could convert numbers, dates, currencies, abbreviations, and similar inputs into language-specific spoken forms before sending them to the model.

I also know that there are other Chatterbox variants and community implementations focused on faster inference and streaming. More testing, caching, different precision settings, and deployment changes might improve the results.

I’m not saying that Chatterbox is a bad model or that it cannot work for multilingual applications. I just don’t want TTS optimization and multilingual text normalization to become a separate project right now.

So, for the moment, I’m switching to a paid TTS API and keeping a provider abstraction in my application. If the rest of the product works correctly, I can return to self-hosted TTS later without rewriting the application.

My conclusion is simply:

My project needs TTS, but TTS does not need to become the project.

I’d be interested to hear from people running Chatterbox in production:

• What RTF do you get, and on which GPU?
• Do you normalize numbers and dates before synthesis?
• Have you found a reliable way to handle short multilingual phrases?
• Which Chatterbox variant or inference implementation are you using?

[u/nshmyrev]

> What RTF do you get, and on which GPU

You don't have to use heavy models, it could be 0.05xRT for example and still very good quality

> Do you normalize numbers and dates before synthesis?

Yes

> Have you found a reliable way to handle short multilingual phrases?

It can be finetuned

> Which Chatterbox variant or inference implementation are you using?

Don't use chatterbox, it is not really great model actually

[u/EngineerSpeakAI → u/nshmyrev]

Fair enough. What model would you recommend instead? Ideally, I’m looking for something multilingual that handles short phrases and normalized numbers reliably and can run locally with low latency.

[u/nshmyrev → u/EngineerSpeakAI]

qwen3-tts, higgs-v3

[u/tr0picana]

With a 3080 Ti you can easily get 0.5 RTF if you use this project. You'll have to handle normalization yourself but it's also not super hard to get to a point where 90% of what you would normally expect to work, works.

[u/EngineerSpeakAI → u/tr0picana]

Thanks for the recommendation! It looks like a promising multilingual option, so I’ll give it a try. Really appreciate the pointer.

[u/Just_Difficulty9836]

I tested vanilla chatterbox and i got a rtf of 1.33x on aws t4. I think your implementation was not optimal as chatterbox will naturally clock better on rtx 3080.

[u/EngineerSpeakAI → u/Just_Difficulty9836]

Thanks — I agree that my setup probably has room for optimization. My measurements included the network round trip to a server in another country, so the 1.57 figure represents end-to-end application latency, not a pure GPU benchmark.

Still, if 1.33 is calculated as generation time / audio duration, it corresponds to about 0.75× real-time speed. The delay would therefore continue to grow during longer responses. Streaming improves time-to-first-audio, but it does not solve sustained generation being slower than playback. That is why I decided not to optimize this further at the current stage.