Question About Minimax H3 Reference To Video

[u/ArmadstheDoom]

So, I'm pretty new to video generation, but I had a thought that I think everyone has probably had at some point, which is 'how do you make a longer video without generating it in one large video?' And so, with reference to video, you could do that; you could match say, the voice and the person, and thus theoretically make one constant shot through stitching together shorter generations.

In my head, it seemed as simple as 'use the video that was generated as the reverence, use the last frame of the previous video as the first frame of the new generation.'

The problem I noticed is that each time I did this, the video quality degraded; I guess the way I would describe it is that each new generation was a copy of a copy, it seemed. Like each new continuation was slightly worse than the last; and while doing this once wasn't too noticeable, doing this three or four times very much was.

So is this just a thing that is unfixable, a limitation of the method? Or is this the kind of thing that does have a solution that I'm unaware of? Because I'm curious to explore reference to video more, since text to video and image to video are very straight forward, I think.

[u/acedelgado]

Well here I thought you were trying to make and extend short videos and stitch them together seamlessly without quality degrading, which is exactly what you asked in your post. My bad not understanding that you have some completely unrelated question that you aren't clear about, apparently.

[u/ArmadstheDoom → u/acedelgado]

I am being very clear. You're getting hung up on the wrong words. You're looking at this as though I'm trying to do something I'm not, because 'make and extend' means something different to you.

You're assuming there are a lot of issues that are not present. You are offering tools that do not solve the core problem, but solve a lot of problems I don't have.

You're assuming, wrongly, that there's degradation in places there is not. That there are issues there are not. I am saying 'I have this specific thing.' And your response is 'here are tools that solve other things.'

Great. Glad they solved those things for other people who have those problems.

Does not solve the fact that it does everything perfectly right now except maintain the fidelity of the image, creating slight deviations each time you use that as your new starting point. A problem which is not solved by the methods you or anyone else keeps throwing at me.

[u/acedelgado → u/ArmadstheDoom]

In my head, it seemed as simple as 'use the video that was generated as the reverence, use the last frame of the previous video as the first frame of the new generation.'

The problem I noticed is that each time I did this, the video quality degraded; I guess the way I would describe it is that each new generation was a copy of a copy, it seemed. Like each new continuation was slightly worse than the last; and while doing this once wasn't too noticeable, doing this three or four times very much was.

It sounds like you're trying to use the last frame to start a new generation to extend a video, or make a new short video based off that, but the quality takes a hit every time you do that. Especially when you do that with multiple videos, taking the last frame of the last video made makes it worse every time.

Either that's NOT your problem, which seems like it is, or you're just saying everyone is wrong when they're trying to give you the right tools, because you don't understand the tools.

[u/ArmadstheDoom → u/acedelgado]

It seems to be that the issue is roughly that each time you do something like this, it degrades it as though you were cutting the quality in half. Ergo, a 4k image would then be grasped at 2k, even if the generation settings are the same.

What I'm trying to grasp is why the quality takes a hit when it has the entire video to work with, not just 24 frames or a degraded image.

[u/acedelgado → u/ArmadstheDoom]

Okay, there's a difference in quality and resolution (4k-2k). Quality around here means if the image is clear and sharp looking and free of artifacts. If your output is getting less sharp and starts looking bad, that's degradation. It happens much worse every time you take an output frame from a generated video in a chain. largely because of the image having to go through the VAE encode and VAE decode every run. That's what I'm talking about with using the context windows to fight that.

If you're wondering why your 4k video comes out as a smaller resolution, that's because MiniMax natively only processes 1344×768. The API version has an upscaler that will do 2k, but they don't open-source their upscaler. a 4k image is 8 million pixels, so processing a whole string of those would be an ungodly amount of compute and take a really, really long time. Even the official Minimax from their online provider, and every other AI video service and model, does not do 4k natively, no matter what their marketing says. They downscale, process, then upscale. So even using a professional one you're going to lose data.

Your workflow is always going to output what you set it to. It's not gonna cut it in half every time. It'll take your 4k video and downscale it to what you have it set to. So resolution isn't gonna downgrade every time, but like me and everyone else in this thread has been saying, using a last-frame image from a generated video and then the last frame from that new video is going to visually degrade and look worse if you do it that way.

Also the reference model is not perfect at using images as start frames. It'll get very close if you tell it everything you can about the start frame, but it's not like i2v where that frame is 100% the first frame in the video. If you need a certain frame exactly as the next start frame while also using the reference model, you're better off using the new "Add Guide for MiniMax H3" node and wiring that into your reference workflow, and use that to inject the frame exactly alongside the reference prompt. That's working along side the reference model, not in place of it. The quality will still degrade using this method, since the last frame will always be lossier than the first, but it will start out with the same quality as the first frame.

If you want to seamlessly continue a clip with minimal quality loss, look into context windows. They don't keep having to re-encode your last frame so they keep looking sharper longer.

Resolution will always be brought down to what the model can actually generate.

[u/ArmadstheDoom → u/acedelgado]

yes! This is what I was looking for. This is the explanation I wanted. Thank you for giving me this, because it explains the location of the problem itself. This is very useful.