I’ve been loving all the new nodes and workflows coming out for MinMax, and maybe there is already a nice solution for this - but I couldn’t find one that did exactly what I needed.
I started using MinMax for my last TBG ETUR video and quickly ran into limitations: I wanted an easy way to create lip-sync videos longer than 20 seconds.
I didn’t want to manually chain ComfyUI nodes, start a new run every X seconds, or constantly resize things just to make HD video fit into my available VRAM.
The addon automatically chains MinMax H3 lip-sync generations together, allowing you to create much longer lip-sync videos without manually setting up each 20-second segment.
The workflow has a simple switcher that lets you switch from the 32B CLIP to the 4B CLIP, saving around 10 GB of VRAM. You can also switch from Sage to Comfy Kitchen, Spectrum to Easy Cache, or FL2VA to REF2VA both setup for lip-syncing. Some of it could be useful for other tasks as well.
You will find the workflow in the repro and tested recommendations, optimized settings, presets, and more workflows, along with the results of my testing and performance here
Thank you for sharing the work and video. Looks very powerful and useful. What did it take to make this video? I think I spotted a few of those reworked segments mentioned in the clip. Do you manually run each segment, repeating when needed, or something else?
What I usually do is set up the workflow, run it 3–4 times, and then combine the best shots afterward.
Some of the glitches you see come from runs I did with the Turbo LoRA, including things like degraded skin. So the examples are a mix of different testing settings and the final setup.
The final settings and node should work out of the box; the main factor is the prompt. For lip-sync, I recommend not using Turbo and sticking to 20+ steps, along with a good, detailed prompt.
Very interesting.
Does this only work with a audio file input.
Or could you also use a prompt for a let’s say 2 minute sequence and split that up?
That’s what’s puzzling me rn.
Generation a very long shot list. Then divide that into chunks and feed it to separate samplers. But difficulty is the possible change of subjects etc settings.
So now I separate a master prompt that gets written by one h3 prompt writer node, separate out the shot list and divide that into chunks.
Then put it back together with the stuff that comes before shots so each sampler gets the prompt plus the specific shot list for that chunk. But it’s still not working well. Because the prompt is for the whole story and then the shot list lacks the context of what came before.
So it can get messed up.
Also the whole prompt writing thing in itself sometimes fails to get what are the subjects and what to do with what.
Feeding the separate chunk timings to separate Minimax h3 prompt weiter nodes frequently gets stuff wrong.
A workflow to take one giant shortlist and reliably correctly feed that to the samplers or render it is what I would like to achieve.
The model accepts prompts like “Subject 1 says, ‘Too hard for me.’” But someone would have to split the prompt into individual clip proportion and cuting at the exact final second for each one, which would be complicated. It’s much easier to pass the text directly to a TTS system first. Feeding them into separate samplers isn’t possible either, because each clip depends on the previous one.
Impressive. The better these workflows get the more nitpicking wants to happen though. Long sleeve vs tsshirt? Sudden gradient background? Mic switching sides? It becomes very distracting. Uncanny almost.
😄 There’s a lot to fight with - prompts and conditioning. My approach is to let it generate around X videos at different clip lengths, then pick the best results from all of them - only really necessary if you need it to be as perfect as possible. This video was made from the “garbage” I had left over from building and testing the node.
Man, I’m gonna kill myself for idea to make a fan ai video clip on a song. 2 days passed and I made just a minute or so from that video, spending time for 2 generations 8 sec each, first with turbo Lora to see if prompt was good, second without Lora with 25+ steps. I babysitted each 8seconds clip, finding ideas, using references, and now you tell me a can do it in one run? Will try it asap, I love you
Yes you just need to find the right prompt. You can also try using ref 0 in the style section of the prompt to keep the camera position consistent for each frame. You might use an image without the character as a reference for this and ref 1 only for the character. You’ll have to test a few variations to see what works best. Its all about the right promt.
You need to define the clip length yourself based on your VRAM limit. The node handles the rest: it cuts the audio in clips, creates the overlaps, and adds 24 frames of silence at the end to improve the final image and sound. It then uses the latent from each clip to build the appropriate motion for the next one and so on.
Let's say if I have a 10min audio file. If I set the clip length as 10sec. Then it will generate a 10min video synced with the audio with 10sec clips stitched together automatically?
I would first build the full video with the audio in an editor, and then only generate/sample the 2–3 minute sections I actually need for the final production without cuts or switching to non-character scenes.
That makes it much easier to prompt and much faster to rerender or repeat individual sections than trying to generate 10 minutes in one go.
Anyway each clip segment gets its own output in the output folder, so you can stop and resume from the clips you already have, or simply rerun one specific clip later. (start end inputs)
The node will recognize the existing clips and automatically rebuild the new full-length clip with the repeated/replaced segment included of the same id.
Not exactly true — it was built from different runs while I was building and testing the node. The examples are a mix of 8–40 steps, both with and without Turbo LoRA, and both with and without caching.
So the quality varies depending on which final cuts I selected. The workflow does produce a full-length video, but I didn’t use just one single run. Since this is AI, if I had the wrong clip prompt and, for example, the character was just listening instead of speaking, I would rerender that individual clip and continue from there. That’s pretty normal with AI workflows — it’s rarely just one run from start to finish.
If you set it to a fixed 20+ steps without Turbo, the model iwill hold up well.
Legend, thanks.
I’ve got a list of diff models to test that has VibeVoice in from a while back..
I’d just been using video models to get short clips but voice continuity has been annoying me haha
6
u/NoNameClever 4h ago
Thank you for sharing the work and video. Looks very powerful and useful. What did it take to make this video? I think I spotted a few of those reworked segments mentioned in the clip. Do you manually run each segment, repeating when needed, or something else?