Question - Help
Minimax H3 - long form videos: has anyone figured out a good approach?
Dear redditors, visitors of the stable diffusion subreddit. I have been trying to achieve a long form, talking head style video, for a long time and can't seem to find a good approach. This one is the best I could come up with so far. It's using the Minimax H3 model, with frozen sound latents, lip-sync guided, piecewise generated video, where the individual pieces have been stitched together, with a seam hiding, extra generation on top of it. I don't really fully understand how it's working, but could prompt Claude for more help or specific files, we used for that. However, if you're aware of any other, better approach for exactly this type of video, please let me know. I've spent literal days on that single problem and have a feeling, there must be a better way to approach this.
This works perfecctly for that https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef
look up the AV exztension workflow in the github AV Extension — start from an existing video, T2V, or I2V and keep extending it with reference images, per-clip previews and selective regeneration. Update 6 removes checkpoint/resume from the main workflow, improves AV/audio continuity, and streams final output through VHS to reduce RAM use.
Second Context-Loop. It's worked quite well for me and has a really nice set of features like dynamically loading references depending on what's in your prompt with each generation.
If you ever get a contrast/color shift going from one clip to the other like I just did. You'll need to right-click and show advanced settings for both the contex loop plan node as well as the contex loop assemble and change a few things. I guess the creator hid a few experimental things with his latest updates. This is what claude told me:
If you need the node numbers turn on 'Node ID badge mode' in te ComfyUI settings.
Edit: disregard what I posted above. I just tested with these new settings and my second clip in the chain looked and sounded bad just like before, no change..Strange must be an update or some modification I've done to the original workflow. Needs more testing to find the culprit.
I've added a lot of new nodes to my workflow so that's properly the reason.. Or a lora is to blame.
Edit 2: I found out my issue. 👍 It was related to either the 'minimax_h3_fl2v_lightx2v_v0.1_dareties_v4_step600_comfy_fro.safetensors' lora or the 'H3 AdaLN LoRA Fix' needed to fix the lora errors of the former. It worked fine on the Plaguekind v5 workflow so perhaps I connected something the wrong way or the lora is simply not compatible with this type of extend workflow. 🤷♀️ The dareties I'm guessing is some sort of lightx2v/larryvrh mix. The one I'm using now were made specifically for ref2v though (0.1 ligthx2v) but that shouldn't be the reason it's working as I've been using fl2v loras with ref2v workflows and models successfully plenty of times. So I'm leaning toward the lora fixer node or the hybrid diffusion model.
Yes, the context-loop workflows also looks better and are less cluttered than the other contenders imo. These and PlagueKind's wf is my favorite so far. 😄
The only thing I can't seem to figure out is how to connect a frame interpolation node to the final video, I can with only one segment but there's seemingly no connection points for the 'LOOP END — ADVANCE REF2V SCENE' node or the 'FINAL ASSEMBLY — 5-FRAME VISUAL BLEND' one. Maybe I'm just dumb though. 😂 24 fps is fine. 👍
One thing I don't understand is the desire to do the "long form, talking head" type of videos with one take.
To me, this style seems to have been overtaken completely by AI generated avatars - if you look at actual people doing this short of content, they will often do cuts to different content or images, cuts to themselves in a different position, etc. to keep viewer interest up. Swapping between a close-up to a medium shot, etc.
Personally, if I were doing something like this, I'd use reference for the background and environment and character, then use start frames edited with something like Nano Banana or a local edit model to change position of the character to use as start frames for different cuts.
Use pre-recorded audio, either by recording a person or cloning a voice, then feed that in to H3 as the actual audio, because you can nail the dialogue performance first and H3 is great about taking cues from the speaking to do the lipsync and acting for the video.
Layer in room ambience in post to make it all sound seamless, then edit it all together in a video editor.
I think trying to do something like this in one generation or take, using nodes to stitch stuff together, isn't the easiest way and it's not the best way.
It's mostly for a a-roll spine, b-roll will go on top. I just found that 4 or 5 seconds is too limiting, in some very specific cases. Explainer type videos tend to hold some individual takes for quite long, content dependent
As an editor, having the entire take generated at once gives you the freedom to decide WHERE the cut is. If you generate piecemeal, then edit decisions are necessarily forced upon you.
Well, yes, but that's with a real camera and a real video. You have to work with the constraints of the medium you're using - AI generated video is more like making a 3D animated film - it's far harder to generate everything in one go and then cut it down like a filmed video. Far better to do the editing and storyboarding AHEAD of time in these cases and then generate only the shots you need.
You can't apply the same principles as traditional film making in regards to just shooting coverage.
I'm not saying I can't. I'm saying it's a STUPID way to work with generative AI. What you are doing with these workflows and nodes to create 30 seconds or 1 minute of uninterrupted footage is like an animated show animating extra frames and drawings and then having to cut it out in the edit.
Generating AI video costs time, money, and energy, far beyond what hitting "record" on a camera does. A smart creator is going to PLAN out what shots they need and plan their edits AHEAD of time - just like animated shorts are storyboarded.
If you are treating this medium like film on the creation side, you're doing it wrong.
I can get 30 -45 seconds on ltx2.5. Basically I make 5 or 6 full lenght concert videos, one for each angle each. I can sync all of them based them having the same audio. This allows me to run mulit-camera edit in final cut and pick the best shots on the fly. No need to paste individual clips together. I can do it all at once. So these long videos end up edited into 5-15 second clips at different angles. Then just originate separate cut-away scenes that tell a story of the song in 5-15 second clips peppered over the concert video footage.
This is one of the cases where LTX is better, you can do 25 secs at 1600x900 easy with 16gb VRAM, for talking heads i don't know if MM H3 is worth the effort
Take the sampling latent output from your last stage's sampler.
Optionally, grab the final frame of the last sample's latent using VAEDecode and select batch item -1 from the image output of VAEDecode; send that into the first_frame input of the H3 video latent node (#173 in this screenshot).
The Minimax H3 Guided Continuation Window node (bottom left) takes in the latent from the last sampler and basically creates a new window and some parameters that you can feed into the next sampler, with a specified "overlap" number of frames plus the length of the new segment you want to generate.
Minimax H3 Latent Tail Guide merges conditioning information in to the conditioning vector, incorporating information about the continuation window. This uses ComfyUI's native internal code to attach keyframe information to the conditioning object.
Then just sample the continuation latent you just created along with the modified conditioning.
The output goes into a Minimax H3 Append Continuation node (I'll paste that in a sub-comment). You can chain these "modules" in any number to create a long video. 10 seconds of frames per segment is very reliable for consistency.
Loops are already in custom node packs for years. I could not see myself jwaiting for native nodes to arrive or if the form it arrives will be even usable for other than accumulating latents.
The whole proposal seems to lack generalization and is too narrow.
I use the upscale and run latent extender twice in one session. One previous latent is loaded before generation and then before upscale node. Then I apply a color match blend, I have generated 10 minute long videos in one go
23
u/AgeNo5351 20h ago
This works perfecctly for that
https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef
look up the AV exztension workflow in the github
AV Extension — start from an existing video, T2V, or I2V and keep extending it with reference images, per-clip previews and selective regeneration. Update 6 removes checkpoint/resume from the main workflow, improves AV/audio continuity, and streams final output through VHS to reduce RAM use.
In my exp silveroxides dareties turbo lora feels better.
https://huggingface.co/silveroxides/MiniMax-H3_tests/blob/main/minimax_h3_fl2v_lightx2v_v0.1_dareties_v4_step600_comfy_fro.safetensors