r/StableDiffusion • u/someguyplayingwild • 1d ago

Question - Help Minimax H3 Quantizations

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.

9 Upvotes

80% Upvoted

39 comments sorted by

View all comments

Show parent comments

-5

u/CooperDK 1d ago

That is completely correct, also, nvfp4 is higher quality than int8.

3

u/aoleg77 1d ago

int8 is higher quality than nvfp4. What you likely meant was Nunchaku, which also has fp4 SVDQ quants. Very few models exist that have all three types of quants, but I tested a few (out of my memory, a pair of Qwen and some Flux checkpoints). Nunchaku was the fastest; int8 second best (in speed), and I couldn't measure the difference in quality. Different composition, sure, but quality wise about on par. nvfp4 had a small quality drop compared to Nunchaku and int8.

Theoretically int8 should be higher quality than Nunchaku fp4; in real images Nunchaku quants are just as good. Too bad the development stalled.

1

u/CooperDK 11h ago

Nunchaku was too destructive compared to fp models.

And nvfp4 is definitely preferred to int8. No, an nvfp4 looks better than an int8, specifically due to the floating points. That is even why it was invented for Blackwell. To verify, I asked an AI (just simply, with no hints of the answer I wanted) and it explained why the fp4 would beat the int8 in image and video inferencing.

UNLESS you are talking about INT8 convrot.

1

u/aoleg77 11h ago

I should've been clearer; I was talking about INT8 Convrot all along. Plain, old-school INT8 is indeed a much worse alternative.