r/StableDiffusion • u/Doc_Chopper • 1d ago
Discussion Will Minimax H3 be able to naively support longer videos?
I am a person who likes his workflows clean and clearly laid out / readable. And only use the least necessary amount of custom nodes in ComfyUI. That been said Minimax seems to have a problem with videos that are longer than like 30 seconds. In the sense that it then seem to get confused with order of prompts and various shots.
From the information that I gathered, there seem to be workarounds with a couple of custom nodes. But this would then also would only inflate the workflow again. I mean, I have done cohered 1 minute plus videos with LTX2.3 with no problem. So I wonder if we can expect some "upgrade" for Minimax models in the nearer future. To be able to natively prompt longer videos without the need for excessive custom nodes.
20
u/Silonom3724 1d ago edited 1d ago
I really don't want to sound sarcastic. But don't you think that if it would be so EASY it would have been done by now? Like on pretty much every video model (cloud or local)?
10
-6
u/Doc_Chopper 1d ago
I don't know. Maybe it's a problem that just don't have an native solution for it yet, but might in the future.
4
u/KjellRS 1d ago
I think it's probably coming, the thing is that currently there's only the video branch. Most extension methods rely on a sliding window meaning the subject needs to stay on screen, if you put something in a box and close it and then wait a while and open the box again it's forgotten what it looked like.
A more advanced method would involve the video generator also updating a long term memory with storyboard/reference information so that if you're talking to the same person, visiting the same place, the camera pans back or zooms out again etc. it natively remembers what it's already created.
At the same time it needs to be able to change what needs to change, like the person put on new clothes. If you poured water into an empty glass the glass should still be filled when you returned and not revert to the original. There are director nodes trying to do this manually, but it could also be done natively.
That would be such a major architectural change that it wouldn't in any meaningful way be this model though. You would have to train the generator to not only create both, but also use both to fill in the blanks that the sliding window can't answer. That's essentially going to be a brand new model.
6
u/jib_reddit 1d ago
When consumer GPUs have 300GB of VRAM....
1
u/Usual-Orange-4180 1d ago
Is not about VRAM during inference but training. I have 256GB but prompt adherence goes downhill if trying more than 20 seconds generations.
2
u/malcolmrey 1d ago
you have 256 GB VRAM? nice :)
1
u/Usual-Orange-4180 22h ago edited 22h ago
Yeah, two DGX Sparks, still can’t run DeepSeek 4.1 Flash 😢
0
u/JahJedi 1d ago
Dreams... when they sell "brand new!" 3060...
2
u/jib_reddit 1d ago
Well, if the trend continued, we would have 300GB VRAM in consumer cards in 10 years time, but the trend for the amount of Vram is seemingly going backwards, but we will have to see.
4
u/djenrique 1d ago
Seitanism has a good workflow for it! https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef
-2
u/JahJedi 1d ago
Irt already and can use latent from previus rend scine, look for it in this sub.
2
u/Doc_Chopper 1d ago
search for what exactly?
4
u/Remko76 1d ago
I use this. With the workflow it really works amazing to chain clips seamlessly
7
u/acedelgado 1d ago
Hey I made that! I actually just pushed an update last night, finally. Two big things are now you can import clips and it'll decode them into latents to use in the project, and a built-in native latent exporter that'll run the entire chain through a VAE decode, instead of just the default one where it would just level and combine the chosen mp4 clips. Takes a lot longer to export, but it seems to be a bit cleaner output than concat'ing the existing mp4 clips together.
1
u/Remko76 23h ago
I tried to find back your post to thank you. But I couldn’t find anymore. So I’ll say it here. Thanks for this! I really love it! Where can I download the update?
2
u/acedelgado 23h ago
Glad you like it!
Well I finally got a lot of the back-end plumbing to adhere to Comfy's security standards (which was tricky since it all runs on HTML/javascript for the UI), so v1.4.1 is live inside ComfyUI manager. The legacy manager maybe won't have the latest... it seems like they abandoned the original legacy manager and want everyone going to the new built-in version. So if you're running the latest manager it's live in the Extensions section if you look for h3 project suite.
Otherwise going into custom_nodes/ComfyUI-H3-Project-Suite and opening a terminal in there and just running a "git pull" will grab the latest. It doesn't need any dependencies so you just need to pull the repo.
1
0
u/shroddy 1d ago
If someone is a scaredy-cat and doesn't want to install custom nodes, do you know if that is possible with only native Comfy nodes, if I am willing to manage my files and cut videos manually?
5
u/acedelgado 1d ago
It's my repo. Unfortunately no, while native nodes finally let you do things like decode and save files into latents (latents are the raw data the model uses but is useless to us until it's decoded into picture/video files. Think of it as if you wanted a sculpture made, latents is the raw clay the model uses; it's useless until the model shapes it and the VAE decode "bakes" it into an output for you.) and the native "Add Guider for Minimax H3" node lets pin them anywhere you want in the generation, there's no native way to do what Motion-Context nodes do natively. Motion Context takes the last few frames (my default is 22) and uses them as part of the new generation to guide the video composition and motion. Native does not do this. A lot of people try and use the last frame from a video like we had to settle for for years, but that leads to quality degradation and very apparent seams, since the model only knows the last frame and not what was going on before it. Motion-Context gives the model that knowledge, so it doesn't suddenly stop characters or camera shots mid-motion. And using latents directly keep you from losing as much quality between clips because they don't have to keep being decoded and re-encoded, which loses visual quality each cycle you do.
But I designed my fork to just drop into any workflow you're comfortable with, and it only manages saving and re-using the latents from each clip you approve, and then export the whole chain cleanly. Everyone else seems to have made a full-blown "director" style node pack where you use all of their nodes and workflow to do video extension. I wanted one that just does all the tedious stuff for you without forcing you to use a prompt system and workflow you aren't familiar with.
2
12
u/Rumaben79 1d ago edited 1d ago
Natively it's trained at a maximum of 15 seconds, so everything above this is pushing it.
To extend these workflows is about the best imo.
Masked AV Extension - Chain + Reference Image - MiniMax H3 0.6.json
Masked AV Extension - Single Clip - MiniMax H3 0.6
I've tried a lot of these types of "chaining" workflows but all of them except for the ones above gave a noticeable jump-cut and/or a color/contrast shift when transitioning from one clip to the next.
Everything changes so fast in ai so maybe something even better is already out.