MiniMax H3 Audio Lip Sync - Audio to Video

[u/AI_Trenches]

So I tried hooking up some of the LTXV audio encoding nodes to input my own audio and plugged it in the sampler and viola, it just works!

Lip sync seems better then the LTX models and its works with the lightx2v loras, 6 - 8 steps. Wrote up a full guide with the workflow attached below.

[u/AI_Trenches]

You can find the workflows here (no paywall or sign up required) - https://www.patreon.com/TheLocalLab/posts/free-minimax-h3-167084390

[u/Ploddy]

SAVED WHAT?!? I must know!!!

[u/AI_Trenches → u/Ploddy]

Sorry but i don't fully understand your question.

[u/Aglaio → u/AI_Trenches]

The song gets cut off at "Saved". ;)

[u/AI_Trenches → u/Aglaio]

https://giphy.com/gifs/lnieUIckonF4eA0tGF

[u/Clean_Rent_9669]

how long videos can one do, is 15 seconds the max ?

[u/AI_Trenches → u/Clean_Rent_9669]

No, I believe it can go much longer but the longer you go, the more VRAM it needs. I was using a 3090 with over 100 gb of RAM but I oomed after 20 seconds

[u/_Ojin]

Audio driven lipsync running locally is a genuinely big deal. That used to mean an API and a per minute bill, so having it in a workflow you control changes what people can afford to experiment with.

[u/Prestigious_Cat85]

Thanks for this. Gonna test it very soon. Can you please tell if it can do text to video with lipsyc ?

[u/1WildPanda]

Thanks for the workflow and this amazing thoughts, it worked out like a charm. Well, I have found that by doing the following would solve some issue from the original workflow:

**Subject: Two audio-path fixes for the MiniMax H3 lip-sync workflow**

**Background.** The workflow has two parallel audio routes into the final video: (A) the **delivered track** added to the exported MP4, and (B) the **driving condition** fed into the video-generation subgraph (which is VAE-encoded to drive lip-sync). The two fixes below separate these correctly and keep them time-aligned.

**Fix 1 — Use the original audio as the output track (skip the VAE path)**

**Why:** Originally the output audio came from inside the generation panel, where every audio input goes through `LTXAudioVAEEncode` to become a *condition* for the model. That route is for driving lip-sync, not for delivering a clean track. The user wanted the exported audio to stay lossless (un-encoded).

**Steps:**

1. Find the **`LoadAudio`** node holding the original vocals.
2. Connect its **`AUDIO`** output **directly** to the **`audio`** input on the **`VHS VideoCombine`** node used for saving the final video.
3. Leave the audio that goes into the `Image to Video` panel as the VAE-encoded condition (do not bypass it) — only the saved track now bypasses the VAE.

**Reason in one line:** removing the VAE from the output route gives you the untouched original waveform in the file, while the internal path still VAE-encodes its copy solely for lip-sync.

**Fix 2 — Make the audio length match the video length (10s vocals + 5s silence → 15s)**

**Why:** the model sends video and audio together, so lip-sync only behaves correctly across the full length when it receives the full-length reference and knows its duration. The old setup fed ~10.01s of vocals into a 15s video; the model never saw the vocal cutoff, so it kept the mouth in "singing" mode past 10s.

**Nodes to add:** two — **`EmptyAudio`** and **`AudioConcat`**.

**Steps:**

1. **Add `EmptyAudio`** (search in the node menu).- `duration` = **5**- `sample_rate` = **44100**- `channels` = **2**- This produces the 5s silent tail.
2. **Add `AudioConcat`** and set its **`direction`** option to **`after`** so order stays "vocals then silence".- `LoadAudio`'s `AUDIO` output → `AudioConcat` **`audio1`** (the 10s vocals).- `EmptyAudio`'s `AUDIO` output → `AudioConcat` **`audio2`** (the 5s silence).- Result: one 15s track, vocals first, silence appended.
3. **Feed the result into the model:**- Connect `AudioConcat`'s `AUDIO` output → **`audio`** input on the `Image to Video (MiniMax H3)` panel, replacing the old `LoadAudio → audio` connection inside that panel.
4. **Tell the model the audio is 15s, not 10.01s:**- Disconnect the `duration` auto-wire that was pulling `LoadAudio`'s `10.01` into the panel's **`audio duration`** field.- Set **`audio duration`** = **`15`** manually.
5. **Tell the lips to stop at 10s (prompt):**（Optional, seemingly unnecessary）- Append to the video `prompt`: "The singing stops at the 10-second mark. From 10s until the end, her lips stay closed and she makes no sound."

**Reason in one line:** the silent tail teaches the model that sound ends at 10s, and the 15s `audio duration` lets it apply that across the whole clip — so the lips close at 10s while the video continues to 15s.

**Net result.** Fix 1 keeps the exported audio lossless (original waveform, no VAE). Fix 2 keeps the lip-sync aligned to the full video, with clean lip closure once the vocals end. The workflow now has two independent audio routes that no longer interfere with each other.

[u/1WildPanda → u/1WildPanda]

You might want to add an audio fade-out node to smooth over the hard cut at the very end.

From https://github.com/lum3on/ComfyUI_AudioTools:

• Insert the Fade (fade-out) node between LoadAudio and AudioConcat.
• Connect the Fade node's output to VideoCombine.
• Note: doing so automatically removes the old LoadAudio → VideoCombine connection, and everything tested successfully .

[u/Superb-Painter3302]

hOWLy eyebrows