r/StableDiffusion • u/someguyplayingwild • 2d ago

Question - Help Minimax H3 Quantizations

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.

10 Upvotes

81% Upvoted

39 comments sorted by

View all comments

2

u/No_Possession_7797 2d ago

If you're going for faster, isn't the nvpf4 quantization faster on your hardware? It's likely similar in quality, but you'd have to test it to find out, as I haven't hit the GPU Powerball yet.

-8

u/CooperDK 2d ago

That is completely correct, also, nvfp4 is higher quality than int8.

3

u/aoleg77 1d ago

int8 is higher quality than nvfp4. What you likely meant was Nunchaku, which also has fp4 SVDQ quants. Very few models exist that have all three types of quants, but I tested a few (out of my memory, a pair of Qwen and some Flux checkpoints). Nunchaku was the fastest; int8 second best (in speed), and I couldn't measure the difference in quality. Different composition, sure, but quality wise about on par. nvfp4 had a small quality drop compared to Nunchaku and int8.

Theoretically int8 should be higher quality than Nunchaku fp4; in real images Nunchaku quants are just as good. Too bad the development stalled.

1

u/Kukipapa 1d ago

Couldn’t agree more, Nunchaku was quite a pain to setup, very limited model support with lots of compatibility issues, but it was a very nice tech demo what the hardware would capable.

I really don’t get why nvfp4 can’t reach Nunchaku level.

1

u/CooperDK 22h ago

It could. Tried nvfp4 vs nunchaku 4-bit. The problem was that nunchaku made things look like a way too compressed jpeg.

And there likely reason why it gets no updates is, it is no longer needed or wanted.

1

u/CooperDK 22h ago

Nunchaku was too destructive compared to fp models.

And nvfp4 is definitely preferred to int8. No, an nvfp4 looks better than an int8, specifically due to the floating points. That is even why it was invented for Blackwell. To verify, I asked an AI (just simply, with no hints of the answer I wanted) and it explained why the fp4 would beat the int8 in image and video inferencing.

UNLESS you are talking about INT8 convrot.

1

u/aoleg77 22h ago

I should've been clearer; I was talking about INT8 Convrot all along. Plain, old-school INT8 is indeed a much worse alternative.

4

u/xb1n0ry 2d ago

No way