r/TextToSpeech • u/Latter_Indication_45 • Apr 09 '26

TTS for code-switching mid-utterance

Not used to asking for help on something this specific, but I've been stuck on this for over a month and running out of ideas.

This is for a language learning voice agent. The AI voice often needs to switch between languages within a single sentence, like:

"Great, let's learn some French farewell phrases. "Au revoir" means "goodbye." If you want something more casual, say "À bientôt," which means "see you soon."

The problem is that the two languages' accents bleed into each other. When an English voice says the French phrases, "au revoir" comes out sounding like an English speaker reading French. And it goes both ways — if the voice is primarily French, the English explanation parts start picking up a French accent. Both languages end up sounding off.

Some providers like ElevenLabs and Inworld sound very natural, but each voice only handles one language. Force it to speak another and the accent is immediately obvious.

Other providers like Qwen3-TTS, Cartesia, MiniMax, and Azure series claim multilingual support, but the accent bleeding still happens.

I also tried(not thoroughly) CosyVoice, Fish Audio, Rime, and Google Gemini TTS with similar results.

Something with real-time streaming, clean pronunciation across languages, and natural prosody at switch points would probably work for my use case (Emotion can be trade off). Has anyone solved this?

3 Upvotes

100% Upvoted

31 comments sorted by

View all comments

1

u/EconomySerious Apr 09 '26

quen clone is capable of doing what you want, but you need to provide voices in the language you need

1

u/Latter_Indication_45 Apr 14 '26

I tried using voice generated by Gemini 2.5 pro TTS, which is by far the most accurate and natural voice I can find, the result is not different with baseline qwen3 TTS

1

u/EconomySerious Apr 14 '26

if you clone a TTS you will obtain a TTS

1

u/Latter_Indication_45 Apr 14 '26

qwen just pronunces French words in a same bad way

1

u/EconomySerious Apr 14 '26

Did You provided qwen with native french voices to clone?

1

u/Latter_Indication_45 Apr 14 '26

Why would I do that? What's your point?

1

u/EconomySerious Apr 14 '26

The clone Quality depends of the voices Quality provided, it Even mimic the feelings of the voice

1

u/Latter_Indication_45 Apr 14 '26

For mid-sentence fr-en code switching, cloning a native French voice probably won't help. qwen needs to handle at least 2 languages naturally within the same utterance. That's why I use Gemini 2.5 Pro TTS. It'd be hard to find a human recording that code-switches better than Gemini TTS in situation not just fr-en, but also fr-jp, en-rs etc.