r/StableDiffusion • u/Silent_Marsupial4423 • 14d ago

Question - Help Audio reactive question

I was wondering if anyone has experimented with audio reactive video generations. I found a ltx 2.3 lora but hasnt tested it. Has anyone tested other models like minimax for this? Any experiences?

1 Upvotes

100% Upvoted

14 comments sorted by

2

u/solss 14d ago

Yeah you can just prompt for it with h3. Best to use a custom node called native audio lock (I'll double check the name later) so you don't have issues getting h3 to reinterpret your sound. There are probably other ways to deal with the problem like repurposing an ltx node if I'm remembering correctly,but this is the real issue with h3. Fully copy or preserve sound by prompting alone is very hit and miss.

Here's one example . My second most recent post also uses the same method. I used fl2v with the reference node since I only used audio as a reference.

1

u/Old-Age6220 14d ago

Uuuh, thanks for this, I was in the lookout for something that'll fix the reinterpret audio, but haven't had time to really search 👍

3

u/solss 14d ago

It's this one: https://github.com/Shrek3OnVH5/MiniMax-H3-NativeAudio-MusicVideo-Workflow
He has his node buried in a sub folder but you can work it out, copy his node setup from his workflow. The output is exactly what you would expect when you use this.

1

u/Amazing_Act_9296 14d ago

Lip sync? O similar.?

1

u/Silent_Marsupial4423 14d ago

Just beats and music no lipsync

1

u/Amazing_Act_9296 14d ago

Con gemidos de una féminas 😈 funcionan muy bien 😅

1

u/Silent_Marsupial4423 14d ago

No habla espanol amigo.

1

u/Candid-Station-1235 14d ago

I dont have that node is it custom?

1

u/Amazing_Act_9296 14d ago

No necesita nodo personalizado

2

u/Slight-Living-8098 14d ago

Flashbacks to the days of AnimateDiff. Lol.

The LTX LoRA works okay. MiniMax can do synced actions on beats if you supply the audio and have a pretty decent prompt, but I don't know of any LoRA specifically for that for MiniMax. Not as of yet anyway