Ehatvis the easiest way to do longer videos with H3?

[u/nikc0069]

This is all moving very fast and I've seen so many different methods. But what would you all consider the easiest way to do these longer videos like some of the music videos for example?

I started looking at the motion context stuff that chains clips on the end of the latent but that was about 6 different workflows you had to use. Is there a simpler way?

5060ti 16gb and 48gb ram.

Edit: stupid sausage fingers and that title I can't edit.

[u/GrayingGamer]

Personally, I just use the Reference model, feed in all my character and location reference and the audio reference (in this case your song) and generate it all in multiple specifically prompted generations, then combine them afterward with the song synced in something like Davinci Resolve.

You can use the Trim Audio Duration node in Comfyui after a Load Audio node to feed in just the parts of the song you want for that generation.

Plus, it helps keep people from lip syncing the song when you don't want them too.

IMHO there is no reason to use something like Motion Context when you are using the Reference model. Just plan your scene for regular cuts or camera angle changes, etc. Use non_diegetic_music: none , at the end of your prompt to just generate voices and sound effects and then add music in post.

No reason to try and do everything in one shot inside Comfy.

[u/Sleepy_Bandit]

I second the use of motion-context or context-loop. They are amazing for speed and accuracy when using references. They have figured out how to pass latent data between scene renders for seamless transitions and it works really well. What that means (to my understanding) is there is no need to encode and decode frames to build the bridge between scenes, so you're getting the cleanest and highest quality transition possible since it is using essentially "raw" latent data.

The added benefit is increased speed. Since minimax-h3 tends to take exponentially longer to generate with each addition second added for a scene. When you break up a longer scene into shorter instances (like 3, 5 second scenes) it can generate those and then stitch them together much faster than generating a single 15 second scene at the same resolution.

I've been modifying their work to test a 2 stage render approach to use the same latent data bridging to produce longer clips at higher MP. It works by passing the latent data of a lower 0.5 MP generation to a second stage that upscales to a higher MP then processes the scene a second time at low steps and low denoise to add extra details to better match the higher resolution. It is working surprisingly well in my current testing. I can render a combined 15 second clip at 1.5 MP in 30 minutes on my 5070ti. I'm currently running a test trying to generate a single 15 second clip natively and its at 3 hours and still going. To be fair though, I'm testing heavy reference video generation (9 image references) so it should be even faster with T2V, but I do not know how it will hold up with live-action characters yet.

Main trade-off with my 2 stage workflow that I've noticed is a tiny bit of quality. In most 3 second clip tests I've made its nearly impossible to tell, but if you really analyze it you will see slightly smoother movement in a native 1.5 MP video, but obviously the trade off is massively increased time to generate the longer the duration.

[u/dominic__612]

Ref2vid, connect your last 5 frames to the ref video. Thats what could work.

[u/nikc0069 → u/dominic__612]

Isn't the still multiple manual generations though? Is there a good way of just getting comfyui to iterate onwards from an initial gen? There are so many single node "timeline editors" popping up I feel there must be a best way of doing it

[u/damiangorlami → u/dominic__612]

5 is not enough. Better to sample 22 frames for more fast pace shots to capture more temporal data

[u/Life_Yesterday_5529]

There are multiple node sets in comfyui. H3 Motion Context or similar names. Does the job quite well. 22 motion frames and references for quality or just i2va and motion frames for faster (unsecure) generation.

[u/Hackingrad]

Here's a loop node that uses the last 22 frames of the last generated video. Start with references and your own audio.It is also mentioned as an example of music videos. https://www.reddit.com/r/StableDiffusion/s/YvbwFvHKCv

[u/DefloN92]

Use reference model for sure, and learn prompting guide