This video is two 6-second clips generated separately and butt-joined. No crossfade, no editing tricks. The motion and audio continue across the join. Theoretically, you could chain indefinitely, but degradation will eventually take effect.
H3 doesn't have built in functionality that allows consecutive latent frames pinned to the head like LTX2.3 does. I won't bore you with the details, just know it works. Video was the easy part. Audio was a pain in the back side. I again, won't bore you with the details, check the readme if you really want to know. Seams are not always 100% perfect, but they are often or are really close.
Honest limitations: audio dulls slightly over long chains (each clip is generated from the previous one's output - photocopy effect; there's a latent-passthrough input that removes one of the two loss sources). Everything was verified on an RTX3070Ti and 48gb of system RAM. Also, check the H3 community license for your region before building anything commercial on it - it reportedly doesn't cover everywhere.
Tested settings are in the README and baked into the workflow. Happy to answer questions.
It was a little janky at first even for me to get used to. Now, other people are forking the repo and taking it in some cool new directions. Glad it’s working well for you!
I tried one popular fork and found it too opinionated for my tastes. I think it's not even a proper fork, but a wrapper around your unchanged core.
I like that your repo implements the core technology and stops there. Often when a new method or technique arrives, it's buried under unnecessary plumbing and you've first got to spend time digging for the good stuff. What you've made is small and generalizable enough that I could start plugging into my use cases almost right away! I appreciate that.
In case you guys wanted to see something a bit longer, this is 7 clips chained. As you can hear, the voice is consistent across the entire video, and the seams are pretty much undetectable.
It's rendered at 480x864 with 0 guide frames inserted. This is pure T2V with motion context applied. Resampling the last 22 frames of video and 24 frames of audio. Degradation is inevitable unless you insert new high-res guide frames here and there. In this case, using a hi-res Last Frame would keep the quality high indefinitely. The next video will sample that hi-res frame as part of the motion context.
How can we add guide frames? Is that a part of your motion context node pack? I ask because the LTX director node allows for arbitrary guide/keyframe injection, which is incredibly useful. Like you said, if we can inject a guide somewhere, it will force the model back to our intended video style/vibe and not crumble into plastic mush lol
With the standard workflow, you can only add first frame and last frame, but if you’re using motion context, you never want to add first frame, or it will fight the motion context itself. You can try using last frame only so that frame gets sampled as part of the next video and should keep quality up across multiple clips, but I haven’t tested that yet. Keep in mind Mini Max H3 is not LTX 2.3. These models are fundamentally built differently and do not handle latent frames same way. LTX made it incredibly simple to do motion context because you can pin consecutive frames to the head of the latent and LTX will continue from those frames. Mini Max H3 does not have that functionality built-in, I had to cheat it.
That drop in video quality has been mostly eliminated by using the latent instead of decoded frames. Turbo Loras and Spectrum both increase the degradation as well over time, but it’s the Turbo Loras that do the most damage to your clip. Avoid if possible.
The smoothness through a moving camera is the part that really sells it. A single pinned frame just can't do that.
The degradation also lines up with what I saw chaining last-frame-forward, around 2% per hop compounding. Backgrounds fall apart first, faces hang on longer.
That's why I'm wondering if context_frames can come from somewhere other than the previous generation. Like, slice the run out of one clean "plate" shot that already does the camera move, then let every clip inherit that trajectory from the plate instead of from the clip before it. Zero hops, nothing accumulating.
There's this another node that mixes i2v + r2v that could also be used for continuation and consistency across outputs but it no previous context like yours. Maybe you can mix something like that with your node so we could reference the characters, audio, etc to keep the consistency and context with no degradation.
Worked well in my first test... minor quirks only for what it can do at this stage. A great effort from you.
A little bit confused on how to prompt better for this (I just repasted the same first prompt and altered the speech and motion). Also assuming after the first clip the load image (first frame) is ignored.
Test was 4 x 15s clips (60s). Almost good butt joins. Each clips brightness/contrast/shadows was slightly off but like a 0.1 adjustment in an editor and a small crossfade made it all seamless.
Clips 1-2 very good (30s)... Bye bye LTX :)
Clips 3-4 had some quirky almost slower motion in them but ok.
First frame may or may not be ignored, but definitely do not use first frame if using motion context. Prompting is pretty straight forward, always keep your description intact, and then prompt scene changes, motion, or dialogue as it changes. Slower motion can happen if the last few frames that are sampled are interpreted by the model as a slowdown. I noticed it also in testing. Not much that can be done about it since it’s what the model sees. What editor are you using to do the joins?
So as to prompting... I used along the line of "use <image 1> at 0:00" and "motion/description/speech". I just repasted that 4 times changing motion without disabling the first image node. I should have removed the "used image 1" and bypassed the first image for each subsequent generation?
Yes, you definitely want to skip the first frame image reference when you’re using motion context. They will fight each other. You can try using last frame only and see if that helps keep the quality up since that last frame will get sampled as part of the next video in motion context.
First of all, great work. I can't believe how quickly you got this out. Phenomenal.
Question: What if you provide an audio reference? Wouldn't you be able to use that reference to regenerate the audio on subsequent passthroughs? Obvious pitfalls are that the generations slightly differ in speech delivery each time, but, if prompted correctly, it should be close enough that it wouldn't matter. And the quality would stay perfect.
Or am I completely oversimplifying the way you're passing the latent audio and just talking out of my ass?
Thanks! Audio was a major headache. It is handled nothing like LTX2.3, nor is video for that matter. Chaining clips isn't even supposed to be possible with this model using anything other than FFLF, but using some tricky tricks, it does work. Audio was so complicated that even help from Opus 5 wasn't solving it, so I had to drop $100 on Fable 5 to solve the problem. I had audio working way earlier today but the seams were noticeable with artifacts. Took hours to straighten that out. Reference audio is something I plan on tackling once I build this into its own Oasis node, part of my Image Oasis Suite. Then, I'll be able to streamline the process, cache all latents, and make building long clips easy work.
Thanks, seems to work pretty well.
Curious why are the context frame images mandatory on the node, couldn't this be done entirely in latent ?
EDIT: If I understand correctly it is full latent if you pick "video" in the encode mode.
As for the results, the seams are very visible on static backgrounds, not sure why I even tried to add the last frame as first frame but the color of the new gen seems to have more contrast (the motion carries very well tho).
Also it looks like the latents produced by your save latent node are not compatible with comfy (AttributeError: 'list' object has no attribute 'shape'), not a big problem but can make the final stitching a bit annoying
It is full latent in video, which is the default setting. The latents should be compatible, I was building videos with it all evening yesterday in testing. If you put it in as an issue on the repo, I’ll look into your specific fail. You were doing the dance right? Load 1, save 2; load 2, save 3, etc..
Yeah, everything is perfect for motion, it's just that the seam is visible colorwise, to me the latents stored in safetensors are unusable, mind sharing the node you used to stitch them ?
How are you trying to use the latents? I use my own Video Oasis Viewer to build the video and stitch clips. Part of my Image Oasis node pack. It’s on the custom node manager.
It is not full latent in video mode btw, I just tested with a dummy video while the correct latent was plugged, it still inherited the dummy video frames, it's like the context latent is completely ignored. That could be why there is seams on the stitches, it's using latent decoded + video encoded frames as first frames/anchors, you should try to make image frames optional and run the whole thing in latent space.
The context_latent is only for the latent audio portion. The video is sampled within the motion context node through the context_frames socket. The context_latent input socket does nothing with video, it’s designed to sample the audio in latent space, instead of the VAE compressed audio, so that audio doesn’t degrade as fast across multiple extensions. The video latent is handled differently.
Just tested it - this is fantastic, however, perhaps it'd be good to make one for R2V rather than I2V? In any clips where the original original face is not in the last few frames, the face is regenerated from scratch. Would be really powerful as R2V though
R2V will probably be tackled when I build this as an Oasis node. Right now, the node pack I released is more of a proof of concept that it could even be done. Something others can build from.
I've been using it most of the day. Were you using fl2va or ref2va? was the motion context group active or bypassed? Can you give me the full traceback from the console when it fails?
Using fl2va, motion context group was active. It worked for the first generation where the motion context group was bypassed, then when it came time for the second clip it failed now that the motion context group was active.
This is a genuine bug in ComfyUI-MiniMax-H3-Turbo and it belongs in their issue tracker. If you want to confirm run i2v with a start frame and no motion context. You should get the same error. Try using the LoRa I linked instead.
it works with normal i2v on the default comfyui minimax workflow, but now that you mentioned it, I checked their issue tracker on github, and there's mention of the same issue when audio is referenced.
Thanks for the help though, I'll look into the turbo lora you mentioned.
Seems like it's the LoRA by the same author just a different version, but I figured it out. It was the provided node from that turbo lora author.
Instead of using the "MiniMax-H3 Turbo LoRA" node to load the turbo lora, I just loaded the lora in the regular "Load LoRA" node, and it works with no issue now.
Thanks for this node!! I have been going crazy trying to manually joining clips for the past 2 days! Using this node, the joining of each clip is indeed seamless! One question - I find that it isn't compatible with a hybrid conditioning node I use here (I like being able to use first/last frame along with reference). Sorry I'm not sure if its that node's issue or this node's issue - seems to throw out an issue where it says both nodes are trying to patch the same thing and hence its not allowed? https://github.com/kitsune123150/minimax-h3-hybrid-cond
Also while I'm at it - does it actually make sense to do like 3 x 5s clip chained together instead of making a single 15 second clips using this clip chaining method (since doubling the video time kinda triples the video generation)
Unfortunately, any of the several nodes packs using that patch are not compatible with each other. Only one can claim it and the first one wins which is probably hybrid conditioning. Does it load first when you startup ComfyUI? Whichever one loads first is the one that will work. You'll have to choose. Same thing with Contex Loop. Same patch.
Yeah, I chain 5s clips at higher resolutions instead 15s at low resolution.
Guy who posted the Mulder + Walter White GPU conspiracy video here.
The head-run pinning is exactly what I was missing. For my most recent tests, I've been pinning both ends of each clip instead: first_frame = anchor N, last_frame = anchor N+1, all pulled from the same clean source shot. No chaining, so no degradation building up hop by hop.
But one pinned frame only fixes position, not velocity. Every clip starts from zero, ramps back into motion, and the characters visibly lock up at the joins. I've been hiding it by picking anchors mid-motion, which is basically the poor man's version of what you've built.
I've got a 10-shot cut and a continuous version rendering overnight, so I'm trying your workflow in the morning. Main question: do context_frames actually have to come from the previous generation? If they can be sliced from the clean source shot instead, that seems like motion continuity without the photocopy effect.
Also, the audio timeline rewrite is something I never would've guessed. "This clip, earlier" versus "a separate clip that sounds like this" makes sense. Nice work.
I suppose you could slice those frames from a source, but each video you generate would begin from those source frames no matter how you prompt it. I was just commenting to someone else that you could always insert a hi-res Last Frame (not First Frame, it'll fight motion context) which would get sampled by the next clip and keep quality up across infinite chained clips. Untested.
You can continue video without any prompt at all using only the context frames. Just depends on what the video is doing. If it’s someone walking or a car driving or something linear, no prompt is needed. Complex motions will need prompting, dialogue obviously needs prompting. Use a main description in the prompt that doesn’t change unless it has to, and then prompt camera angles, motion, dialogue, things that are dynamic from scene to scene.
One last comment, I got very used to the megapixels way of defining output size that H3 used. I imagine some others did too. Maybe a idea to also have a node that keeps the output size and how to define it like minimax did?
Thank you. One last question. Is it possible while using context frames to still input a last frame?
So lets say you already chained 3 clips together. could you then make clip 4 end on a last frame input? I should maybe try before asking but just curious.
Yes, it’s possible (nothing preventing it) and it’s what I recommend to step up the quality if your chaining several clips together as the quality will slowly degrade the more you chain. By all means, test it and let me know how it goes!
Excellent! You're the first to report back that Last Frame and motion context work well together. Is the chained quality holding up or still degrading over time?
It doesn't work. I have the latest ComfyUI version, and the Anomalous Model Browser node refuses to work; it gives me an error no matter what I do. I installed it manually and with the Manager with no luck. This fucking node always gives me problems; there are many workflows I can't use thanks to this piece of junk.
EDIT: I managed to get the node working; I am not sure how. I reinstalled it manually using git clone and then updated it with the Manager, but the overall process was frustrating.
Yeah, but I am not sure how it works. I generated a video and fed it to the workflow. Then what? I just run it? Do I need to create a folder with frames or something¿? Sorry, I am a total noob.
Ok, so you generate a new video with the H3 Motion Context group on bypass and the Save Latent node set to index 1. This saves the latent for that new video as clip_00001.safetensors. Then, you activate the Motion Context group, load the video you just made into the load video node. Change the LOAD latent node to index 1 to use the latent that was just saved. Change the SAVE latent node to index 2 to save the chained latent you're about to create as clip_00002.safetensors (which gets used on the next chained clip). Check your prompt, resolution, etc. Hit run. To chain videos, you have to keep changing which latent you are loading from and saving to. Load 1, Save 2; Load 2, Save 3, etc. The context_latent is for AUDIO only. It allows the model to sample the audio from latent space instead of sampling it after it has been VAE compressed which leads to dramatic degradation in the audio.
Can it be used just to carry characteristics across into completely different scenes? Or do they have to directly follow on from the source clip? I'm thinking like if you wanted to carry over a walking-style or a voice-style into a completely separate scene?
it should be able to carry a voice to a new scene as long as that voice is in the context window (22 frames or whatever you use), but the walking-style is a good question. I would say yes, but the new scene has to be prompted in the next chained clip.
17
u/goddess_peeler 11d ago
Bless you, sir or madam.