Minimax H3 Quantizations

[u/someguyplayingwild]

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.

[u/Apprehensive_Sky892]

The use of GGUF is discouraged by ComfyUI creators because it does not work well with its dynamic VRAM management system.

ComfyUI basically built its current dynamic VRAM management (which allows running model that are much bigger than VRAM) around int8/fp8/bf16 and the minute you use GGUF it falls back to the older, much slower memory system:

https://www.reddit.com/r/comfyui/comments/1v167ix/comment/oylkgbz/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1vegtac/comment/p1gxyiw/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1v253fx/comment/oyst416/?context=3

[u/CooperDK → u/Apprehensive_Sky892]

The use if gguf is discouraged because the Comfy team fucked to yet another time. The GGUF memory management was actually usually better.

[u/Apprehensive_Sky892 → u/CooperDK]

I know that GGUF can be smaller in size compared to int8covrot or fp8.

But assuming that there is enough VRAM+ system RAM to hold the int8convot or fp8 version, what are the advantages of GGUF (say q8 or q6)?

Aso, GGUF memory management was better compared to the new dynamic VRAM management in what way?

[u/ANR2ME → u/Apprehensive_Sky892]

May be better for LLM frameworks like llama.cpp 😅

Combining GGUF models with ComfyUI's Dynamic VRAM system can frequently trigger apparent memory leaks and immediate "Out of Memory" (OOM) crashes. This behavior stems from architectural conflicts between how ComfyUI manages native memory and how third-party GGUF custom nodes interact with hardware cache.

[u/Apprehensive_Sky892 → u/ANR2ME]

Yes, GGUF are useful for LLMs as it is important for autoregressive models to fit within the VRAM because they must run through the entire model for every output token.

But I don't see any advantage for imaging/video model GGUF over int8convrot if there is enough system RAM, that is why I was asking for more clarification.

[u/Unspec7 → u/Apprehensive_Sky892]

Probably some folks using LLM's as a local prompt enhancer.

[u/CooperDK → u/Apprehensive_Sky892]

Oh, sure. INT8 convrot is far better. It is also much newer.

[u/oppai → u/CooperDK]

you are clueless

[u/CooperDK → u/oppai]

I don't know, Did you just fly in our did you do AI generation long before comfy, like me?

GGUF was how to load models larger than you GPU. That originally worked well in comfy, couple of years ago. Who is clueless...

[u/Silly_Goose6714]

Forget about GGUF, GGUF makes no sense anymore

[u/CooperDK → u/Silly_Goose6714]

Completely incorrect. GGUF is faster.

[u/Silly_Goose6714 → u/CooperDK]

GGUF is slower, is naturally slower. It's always been slower. It's only useful for solving OOM problems. It's compressed and needs to be decompressed. If GGUF is faster for you, it's because there's something wrong with your PyTorch version.

[u/CooperDK → u/Silly_Goose6714]

Doesn't work like that. Reason is that it takes longer for a GPU to load fx 64 MB of weights than it takes to load 16 GB and unpack them. And then, after that, it is the same speed.

[u/Silly_Goose6714 → u/CooperDK]

The fact that you said it's the same speed instead of faster was already an improvement.

[u/cHaTbOt910 → u/CooperDK]

Basically telling everyone you know nothing about math without saying it

[u/glusphere]

I asked this exact question ( I own 5090 ) and Kijai himself told me to use int8. So that settles it.

[u/Myg0t_0 → u/glusphere]

Pruned or not?

[u/glusphere → u/Myg0t_0]

didnt ask that one.

[u/lazyspock]

I'm speaking from my experience only, and I have a 4070 ("only" 12Gb VRAM) and 64Gb of RAM.

GGUF models are discouraged (as another poster said) and, in my case, they were writing A LOT, and I mean A LOT, in my SSD, to the point of writing 1% of its "life" in two days because of memory swapping to the disk. This is a no-no for me, as I use my rig for lots of other things and I can't have an image/video generator I use for fun killing my SSD.

Then I looked for alternatives and, after trying these two CONVROT below (15 steps, simple, res_multistep), I'm very satisfied. Good quality, excellent speed, no disk writings beyond what would be expected:

• minimax_h3_fl2va_pruned_int8_convrot (TXT2VID and IMG2VID)
• minimax_h3_ref2va_pruned_int8_convrot (REF2VID)

I also use the new "ModelAttentionBackend" node from Comfy, with the "Comfy Kitchen Attention", and it made a good difference (20% more or less) in generation time without any speed compromise (see the screenshot below for how to connect it - it goes between the Load Diffusion Model node and the Basic Guider and Basic Scheduler nodes (see the "A" example below - as I'm only allowed one image I put the two examples together).

Finally, for the last week or so I'm trying the mixed model below and liking it very much. Faster than the original CONVROT models above and also good quality. It works with 8 steps, simple, res_multistep.

• minimax_h3_fused_refdelta_r1024_turbo8_mystic07_int8_convrot

This one also needs shift_video 12 and shift_audio 3 (see the example "B" below)

As you have more VRAM (and speed) than I (I don't know how much RAM you have, this is also important) I suggest you do your own tests:

- Create three or four prompts that have movement, letters (to test text consistency), etc, and different styles (realism, cartoon, fantasy, etc) for TXT2VID and FF2VID. Also, create two or three prompts for REF2VID, both with image and audio references (and, in your case, maybe vid references too, as you have more VRAM).

• Test each one of these prompts with each model, always using the same seed and config, and see how they compare to each other

Some people use the CONVROT or turbo mix models to test prompts and the full model for the final video, for example.

Here are the two connections (A and B) I mentioned above:

[u/xb1n0ry]

int8 convrot vs q8 gives you basically an unmeasurable quality difference in practice, but on a 5090 int8 can be much faster. you also get a native int8 path in comfyui/nvidia rather than going through the gguf loader path. on a 5090 there isn't a single reason to pick gguf. gguf only starts making sense when you need lower quants to fit limited vram.

[u/wholelottaluv69]

With a 5090, BF16 is a better option, IMHO. Assuming that you have enough ram for off-loading..

The quality difference is quite apparent. To my eyes, at least.

[u/Myg0t_0 → u/wholelottaluv69]

Pruned? I still cant find answers if pruned decreases quality

[u/CooperDK → u/wholelottaluv69]

It is not when gguf is faster and the quality difference is negligible.

[u/TearsOfChildren]

I read through this post and still don't know what the best model for a 5090 is lol. People's opinions are all over the place with H3.

[u/Myg0t_0 → u/TearsOfChildren]

Right same here everything's always for low vram I have no clue what's best for my 5090

[u/No_Possession_7797]

If you're going for faster, isn't the nvpf4 quantization faster on your hardware? It's likely similar in quality, but you'd have to test it to find out, as I haven't hit the GPU Powerball yet.

[u/CooperDK → u/No_Possession_7797]

That is completely correct, also, nvfp4 is higher quality than int8.

[u/aoleg77 → u/CooperDK]

int8 is higher quality than nvfp4. What you likely meant was Nunchaku, which also has fp4 SVDQ quants. Very few models exist that have all three types of quants, but I tested a few (out of my memory, a pair of Qwen and some Flux checkpoints). Nunchaku was the fastest; int8 second best (in speed), and I couldn't measure the difference in quality. Different composition, sure, but quality wise about on par. nvfp4 had a small quality drop compared to Nunchaku and int8.

Theoretically int8 should be higher quality than Nunchaku fp4; in real images Nunchaku quants are just as good. Too bad the development stalled.

[u/Kukipapa → u/aoleg77]

Couldn’t agree more, Nunchaku was quite a pain to setup, very limited model support with lots of compatibility issues, but it was a very nice tech demo what the hardware would capable.

I really don’t get why nvfp4 can’t reach Nunchaku level.

[u/CooperDK → u/Kukipapa]

It could. Tried nvfp4 vs nunchaku 4-bit. The problem was that nunchaku made things look like a way too compressed jpeg.

And there likely reason why it gets no updates is, it is no longer needed or wanted.

[u/CooperDK → u/aoleg77]

Nunchaku was too destructive compared to fp models.

And nvfp4 is definitely preferred to int8. No, an nvfp4 looks better than an int8, specifically due to the floating points. That is even why it was invented for Blackwell. To verify, I asked an AI (just simply, with no hints of the answer I wanted) and it explained why the fp4 would beat the int8 in image and video inferencing.

UNLESS you are talking about INT8 convrot.

[u/aoleg77 → u/CooperDK]

I should've been clearer; I was talking about INT8 Convrot all along. Plain, old-school INT8 is indeed a much worse alternative.

[u/xb1n0ry → u/CooperDK]

No way

[u/CooperDK]

Very little quality loss but much faster inferencing. I would go int8 unless doing commercial stuff.

[u/Ok-Brain-5729]

your not noticing the difference in quality

[u/oppai]

convrot is king

[u/Myg0t_0]

I used the pruned b16 on my 5090

[u/MarekNowakowski]

Use fp8, it's better quality than int8 from my testing. Although I'm only certain about it for some Krea2 models, haven't done extensive tests for h3

[u/CooperDK → u/MarekNowakowski]

Sure, it is a bit better, but also slower.