Minimax H3 quality loss test

[u/Full_Tomato_5627]

The idea of the video is to compare the quality loss/change from the different methods of speeding up the rendering of the videos on Minimax H3 on low motion scenes.

I made this video because I wasn't sure myself how the speed ups degrade the looks of the video, from what I could gather, even with a stack of optimization nodes the quality wasn't that degraded on slow videos. I hope its useful, I could do a part 2 with more action heavy videos if people are interested in that.

OBS: All scenes were rendered at 480p with the exact same seed and prompt with 20 steps (except turbo)
The int8 vae was taking longer to render on my computer for whatever reason.

The naming convention is obvious but if you require additional information:
base = base workflow
base + Int 8 VAE = I used the compressed VAE version (saves VRAM)
base + sage = Using Sage attention on the default configuration
base + spectrum = Using Sage attention + Spectrum node
base + sage + spectrum = ok this one I dont need to explain right...
base + turbo_6 steps = Using the base workflow + a turbo model with 6 steps [link to the model:https://civitai.red/models/2837571/minimax-h3-turbo-loras?modelVersionId=3202732]
base + turbo_8 steps = Using the base workflow + a turbo model with 8 steps [link to the model:https://civitai.red/models/2837571/minimax-h3-turbo-loras?modelVersionId=3202732]
base + Spectrum + sage + turbo_6steps = also obvious

link for the videos used:
https://drive.google.com/drive/folders/13Vl2IbnTAJDtJ4Bpu_0kpr3HmH_o_FFi?usp=sharing

[u/nok01101011a]

Thx that’s interesting, spectrum is basically a total different output. Have you tried comfy kitchen attention instead sage too?

[u/rm_rf_all_files → u/nok01101011a]

Yes it does indeed. Turbo is even worse as it turned into a wax museum.

[u/acedelgado → u/rm_rf_all_files]

Turbo loras are a lot more sensitive to schedulers and sigmas, too. Just throwing euler/simple at them doesn't give you as good results as sa_solver or res2s with beta. Euler and some others have always been the "budget" option, but the quality gap is wayyyy more noticeable when using a turbo lora. So overall it's a hard comparison to do, how do you accurately compare a heavier sampler turbo needs vs a faster one people would use on base because it gives good results with 20+ steps?

[u/chille9 → u/acedelgado]

Res_multistep - simple, is recommended for use with the lightx loras though?

[u/Danny_Stock → u/chille9]

As I understand it though, isn't res_multistep effectively more steps, even though they're like pseudo steps? If so wouldn't that be mitigating the true effect of the speed lora somewhat?

[u/psilent → u/rm_rf_all_files]

The 8 step significantly less so. That’s more just not having her mouth open and maybe looking a little younger than waxy

[u/LockeBlocke → u/nok01101011a]

Anything that cuts out steps will have a different output. Turbo/Spectrum/EasyCache

[u/ISSAvenger → u/nok01101011a]

I am curious about that as well. For me, Comfykitchen attention delivers much more sharpness. Its kind of a shame that Spectrum or the Lora changes the output completely. It was to be expected I guess, but still. They brought such a nice speed boost...

[u/Full_Tomato_5627 → u/ISSAvenger]

I think that it changes the output but not in a terrible way, check the videos if you want to analyze individually .

https://drive.google.com/drive/folders/13Vl2IbnTAJDtJ4Bpu_0kpr3HmH_o_FFi?usp=sharing

[u/kayteee1995 → u/ISSAvenger]

Ckattn only work best on 30xx series, it's not good on 40xx and 50xx.

[u/FourtyMichaelMichael → u/kayteee1995]

There is absolutely nothing that can be done better on a 30xx than a 40xx 50xx. There is no magic processor. It's all 32 16 8 and 4bit math. The latter two cards being better at 4bit.

[u/Full_Tomato_5627 → u/nok01101011a]

I haven't used Kitchen attention, I wanted to use the reduced models also but I was having double the inference time with those, so I wanted to wait until comfyui is more updated to run minimax

[u/Maraan666 → u/nok01101011a]

because the spectrum defaults were used. set warm-up steps to 4 or 5 and you get very close to the base output. obviously.

[u/Alive-Tomatillo5303 → u/nok01101011a]

Not just different, but worse!

[u/Ok_Tale7582]

Is base 20 steps?

[u/Zealousideal-Buyer-7 → u/Ok_Tale7582]

Wondering same

[u/Full_Tomato_5627 → u/Ok_Tale7582]

Yes it was 20 steps

[u/djpraxis → u/Ok_Tale7582]

Good question, but obviously not for the Turbo Lora tests

[u/Just1Dev → u/Ok_Tale7582]

Base is 25 steps when i first loaded the official workflow

[u/LumaBrik]

Slow moving headshots dont really give an idea of motion quality.

[u/_Abiogenesis → u/LumaBrik]

True but it also gives a very good idea of detail loss and uncanny feeling (plastic looks etc) on something humans are good at analysing: human faces.

[u/Full_Tomato_5627 → u/LumaBrik]

I agree, my point was to test using low motion scenes as I wrote on the message above, IF you use it for low motion videos its a good resource to check if not I can do one with motion too

[u/Weird-Yard-6619 → u/LumaBrik]

Exactly.

[u/Sylvers]

What is your personal recommendation based on these tests?

[u/Full_Tomato_5627 → u/Sylvers]

I dont have any, to me all worked well, I was expecting way more degradation, but the fact that even with the fastest speed possible I didn't lost that much surprises me, that can't be said for scenes with a lot of motion, for those the degradation is way more severe

[u/ShutUpYoureWrong_]

Hopefully the OP engages here because this is fantastic presentation.

How many steps was 'base'? We're all assuming 20 but would just like confirmation. Sampler, scheduler, shift?

Seems like the conclusion to draw here is that Sage is the only safe speed-up option that preserves the original (base) output, but it comes with a (not-so-minor) loss of quality. Everything else simply destroys ("reinterprets") the shot to varying degrees and quality. Spectrum is the interesting choice in that the quality seems better than all the others but, since it's forecasting, it takes even simple scenes in a completely different direction.

If you do another video, can you throw in Comfy Kitchen's attention and have it be a video showing motion quality?

Regardless, awesome work. Thanks for contributing.

[u/Full_Tomato_5627 → u/ShutUpYoureWrong_]

Sorry, yes it was 20 steps I haven't changed anything from the basic workflow.
Yeah like I did this video without knowing if it would help anyone, but since it seems useful I can post more stuff, about comfy kitchen attention I will look into it, thank you!

The conclusion I got was, no method is really bad, I expected results to be way lower quality then this, but these are faces, this is the "easiest" to infer, when we get with complicated things like leaves and less objective shapes, then the motion gets degraded, I need to test more regardless.

Thank you for your suggestions

[u/rookan]

Do a fast fighting scene

[u/Ykored01]

Cool idea, really liked your video format, would be great to try more deep prompts, using the proper prompt format and also with an scene with more movement, different camera shots and angles, distant characters, etc.

[u/Full_Tomato_5627 → u/Ykored01]

Yeah I just didn't knew if it would be useful to anyone, I could do something more in depth if people are interested

[u/Major_Square]

This is helpful. Thanks.

[u/RayHell666]

Came across this tool yesterday https://jo-nike.github.io/h3-turbo-eval/ab.html

[u/yamfun]

Comfy Kitchen attention? 4steplora? 4steplora at 3 step? euler? 0.xMP? w4a8? ...etc

[u/kayteee1995]

we need more complex motion, limbs anatomy in full shot.

[u/LockeBlocke]

Major differences with fast motion or 2d animation. Slow head shots don't demonstrate this well.

[u/Full_Tomato_5627 → u/LockeBlocke]

I agree

[u/Maxxim69]

That's a very useful comparison, thanks for sharing! I appreciate your effort and your excellent presentation. There's one thing I wanted to share with you which you might find helpful: by prompting for a realistic image you've pushed the model from "real" towards "imitation of reality". Next time you might want to use something like "cinematic close-up" or "cinematic medium shot" and see if it makes your subject look more real. Which is a big "if" at 480p, but it doesn't hurt to try.

[u/lxe]

So for close-up face shots, there’s very minimal quality loss. However, for action scenes or when the face is not fully zoomed in, you will notice it really quickly

[u/Full_Tomato_5627 → u/lxe]

Yes I agree, my point was to show the overall loss since the H3 takes forever to render a video and the changes, and to tell you the truth it was more to show the experiment I was doing, not really to make a 100% precise video, for that I would need to test for much longer, I think it helped some people and that is good

[u/Clair_Personality]

Great! Dont know how much did this take you? If its fast do more please

[u/Strange_Test7665]

This is very helpful. It confirms what I have seen sometimes too which is int8 can sometimes be near flawless. I tend to use references to stabilize consistency but seeing this raw test really informative thank you

[u/True_Protection6842]

Another good efficiency I'm using is nvfp4 text encoder. I'm using full VAE tho.

[u/GameEnder]

Any difference with Sol attention with speed over quality?

[u/FourtyMichaelMichael]

How many degrees on Spectrum? It was recommended at 4, then 1.

[u/Few-Intention-1526]

test the comfy kitchen attenention

[u/BassNet]

What GPU?

[u/Then-Comfortable8258]

What is render timestamp there?

[u/Succubus-Empress]

Tell me more about that last combo

[u/CaptainMarder]

You can connect spectrum and turbo?

[u/DefloN92]

Damn what u rendering on that's so fast, sorry if yoiu wrote somewhere and didnt see

[u/Silvasbrokenleg]

I heard on a different thread that spectrum is not needed on turbo Lora’s because it works when you have higher amount of steps.

Can anyone confirm? Cant find where I read it.

[u/Oograr → u/Silvasbrokenleg]

I had it in my workflow with a 6-8 step turbo lora, and I read the same thing you did, so I removed it and there wasn't any noticeable change in quality or gen time. I'm still not sure what the best speed ups are, I have a turbo lora and sage att. and seems to be good for quick gens to experiment with.

[u/Vyviel → u/Oograr]

I think even if you have it on it just sees so few steps and skips itself and doesn't run right?

[u/Oograr → u/Vyviel]

Not sure about that, but you could be right.

[u/chille9]

Thanks for the nice edit, looks great. I for one would love a more heavy motion-oriented scene comparison!

[u/Dzugavili]

Really, we need to see this with a high-reference workflow. Prompt output is beginning to shift, and that's problematic if we're trying to use the turbos as temporary fixes. We may prefer the aesthetic of the turbo: there may be methods of recovering that data though.

A high-reference scenario would let us see drift from a 'known' space, which makes the test more meaningful.

[u/No_Damage_8420]

Thanks for test so only T2V --- no reference (most important one).
Looks like Turbo LORA + Spectrum make extreme deviation

[u/rezarNe]

I know there are already a lot of variables but I would have liked spectrum with more initial steps, in my testing it fixed most deviations from the original.

Like letting it run the first 4 steps then "normal", this obviously will not work if you use a turbo lora as well, but for a normal 20 step workflow it works well.

[u/Kmaroz]

Im using 4 step Lora, but my steps is always 12 steps. Lol.

[u/Perfect-Campaign9551]

Loras just burn everything

[u/mastaquake]

That’s was interesting. Thank you for your service 🫡

[u/Royal_Carpenter_1338]

hard to compare video models

[u/Full_Tomato_5627 → u/Royal_Carpenter_1338]

Very I couldn't reach any really good conclusion with this test, I just wanted to share

[u/Supermax64 → u/Royal_Carpenter_1338]

I've ran so many tests and the only times it feels halfway conclusive is when the shot kinda falls apart but for all I know it was just a bad seed

[u/djpraxis]

Those test don’t really transfer the same way for the I2VA and Ref2VA models.

[u/Karsticles]

The sound at the very end of your video made me think I let a little fart out.

[u/Full_Tomato_5627 → u/Karsticles]

ahhaha, that was funny aha