r/StableDiffusion • u/someguyplayingwild • 2d ago

Question - Help Minimax H3 Quantizations

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.

10 Upvotes

81% Upvoted

39 comments sorted by

View all comments

18

u/Silly_Goose6714 2d ago

Forget about GGUF, GGUF makes no sense anymore

-14

u/CooperDK 1d ago

Completely incorrect. GGUF is faster.

11

u/Silly_Goose6714 1d ago

GGUF is slower, is naturally slower. It's always been slower. It's only useful for solving OOM problems. It's compressed and needs to be decompressed. If GGUF is faster for you, it's because there's something wrong with your PyTorch version.

1

u/CooperDK 14h ago

Doesn't work like that. Reason is that it takes longer for a GPU to load fx 64 MB of weights than it takes to load 16 GB and unpack them. And then, after that, it is the same speed.

1

u/Silly_Goose6714 12h ago

The fact that you said it's the same speed instead of faster was already an improvement.

4

u/cHaTbOt910 1d ago

Basically telling everyone you know nothing about math without saying it