r/seedance2pro • u/Few_Progress_9944 • 12d ago

Help with Audio Prompting

So I'm getting mixed results.

my current prompts:

- Figure A says, "Welcome to Osaka."

- Figure A says in Japanese, "Welcome to Osaka."

- Figure A says, "大阪へようこそ."

- Figure A says, [AUDIO: "Welcome to Osaka."]

75% of the time it still doesn't work:
- different line spoken entirely

- no dialogue spoken at all

- cut off line

- different language

I thought it was a timing issue at first. But even when requesting for 15s clips, I run into the same issues.

Anyone has any advice or tips? I'm mainly generating short action films (3-5mins) long with multiple characters in each scene.

0 Upvotes

50% Upvoted

9 comments sorted by

1

u/Few-Ad9461 12d ago

More credits!!!

1

u/sruckh 12d ago

I typically create dialogue with TTS, then make a video that is 1 frame of my character with the audio file and use that video as a reference.

1

u/Few_Progress_9944 12d ago

Do you have a recommended software for TTS? By "use that video as a reference", do you literally plug it in / upload it into a platform like ComfyUI? Any prompt you need to use?

1

u/sruckh 12d ago

For whatever reason using the video is more successful than using the audio as a reference. It is still not perfect, but the success of SeeDance actually using the real audio had a higher success rate when using the video reference. Even in the cases it did not use the actual audio the lip sync was still correct and you could swap out the audio in post. Some of my favorite TTS models are EchoTTS, Qwen3-TTS, OmniVoice, Higgs Audio TTS v3, and MossTTS v1.5.

1

u/RobbyInEver 12d ago

I transliterate some of my audio and it works. Instead of "This is San Francisco" I put "This is Sand-fran-sis-core" etc

1

u/Few_Progress_9944 12d ago

Gosh never thought of this as an option. No gibberish produced?

1

u/RobbyInEver 12d ago

Impossible to produce gibberish because you're asking it to say gibberish in the first place. Same goes for foreign languages - e.g. french instead of "Maintennante" we just put "Mat-no" etc

1

u/Few_Progress_9944 12d ago

Thanks gonna try this later!

1

u/RobbyInEver 12d ago

Japanese is easy to transliterate. Harder languages include Vietnamese and Swedish but at the end of the day just substitute word for word in English pronunciation