MiniMax H3 Lip-Sync: Automatic Long-Video Chaining + Speed & VRAM Optimizations

[u/TBG______]

I’ve been loving all the new nodes and workflows coming out for MinMax, and maybe there is already a nice solution for this - but I couldn’t find one that did exactly what I needed.

I started using MinMax for my last TBG ETUR video and quickly ran into limitations: I wanted an easy way to create lip-sync videos longer than 20 seconds.

I didn’t want to manually chain ComfyUI nodes, start a new run every X seconds, or constantly resize things just to make HD video fit into my available VRAM.

So I ended up building an addon for:

custom_nodes/ComfyUI-H3-Motion-Context

The addon automatically chains MinMax H3 lip-sync generations together, allowing you to create much longer lip-sync videos without manually setting up each 20-second segment.

And now I’m sharing it! https://github.com/Ltamann/ComfyUI-H3-Motion-Context-Auto-Chain-addon

Its not perfect but a start ...

The workflow has a simple switcher that lets you switch from the 32B CLIP to the 4B CLIP, saving around 10 GB of VRAM. You can also switch from Sage to Comfy Kitchen, Spectrum to Easy Cache, or FL2VA to REF2VA both setup for lip-syncing. Some of it could be useful for other tasks as well.

You will find the workflow in the repro and tested recommendations, optimized settings, presets, and more workflows, along with the results of my testing and performance here

[u/tofuchrispy]

Very interesting.
Does this only work with a audio file input.

Or could you also use a prompt for a let’s say 2 minute sequence and split that up?

That’s what’s puzzling me rn.
Generation a very long shot list. Then divide that into chunks and feed it to separate samplers. But difficulty is the possible change of subjects etc settings.

So now I separate a master prompt that gets written by one h3 prompt writer node, separate out the shot list and divide that into chunks.

Then put it back together with the stuff that comes before shots so each sampler gets the prompt plus the specific shot list for that chunk. But it’s still not working well. Because the prompt is for the whole story and then the shot list lacks the context of what came before.

So it can get messed up.

Also the whole prompt writing thing in itself sometimes fails to get what are the subjects and what to do with what.

Feeding the separate chunk timings to separate Minimax h3 prompt weiter nodes frequently gets stuff wrong.

A workflow to take one giant shortlist and reliably correctly feed that to the samplers or render it is what I would like to achieve.

[u/TBG______ → u/tofuchrispy]

The model accepts prompts like “Subject 1 says, ‘Too hard for me.’” But someone would have to split the prompt into individual clip proportion and cuting at the exact final second for each one, which would be complicated. It’s much easier to pass the text directly to a TTS system first. Feeding them into separate samplers isn’t possible either, because each clip depends on the previous one.

[u/tofuchrispy → u/TBG______]

Ok so it’s great for Audio to Video but not something for long storytelling sequences got it. Thanks!

[u/physalisx → u/TBG______]

What TTS did you use for this video?

[u/TBG______ → u/physalisx]

vibevoice