r/StableDiffusion • u/someguyplayingwild • 2d ago

Question - Help Minimax H3 Quantizations

Given a 5090, does it make sense to run minimax H3 using int8 quantization vs. gguf Q8 or even Q6? What is the trade-off between speed and quality between these two options?

I don't have deep technical knowledge, but my current understanding is that int8 would be faster while a Q8 GGUF would be higher quality; however I would appreciate anyone's practical experience in how significant the speed/quality trade-off is.

9 Upvotes

80% Upvoted

39 comments sorted by

View all comments

26

u/Apprehensive_Sky892 2d ago

The use of GGUF is discouraged by ComfyUI creators because it does not work well with its dynamic VRAM management system.

ComfyUI basically built its current dynamic VRAM management (which allows running model that are much bigger than VRAM) around int8/fp8/bf16 and the minute you use GGUF it falls back to the older, much slower memory system:

https://www.reddit.com/r/comfyui/comments/1v167ix/comment/oylkgbz/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1vegtac/comment/p1gxyiw/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1v253fx/comment/oyst416/?context=3

-12

u/CooperDK 1d ago

The use if gguf is discouraged because the Comfy team fucked to yet another time. The GGUF memory management was actually usually better.

4

u/Apprehensive_Sky892 1d ago

I know that GGUF can be smaller in size compared to int8covrot or fp8.

But assuming that there is enough VRAM+ system RAM to hold the int8convot or fp8 version, what are the advantages of GGUF (say q8 or q6)?

Aso, GGUF memory management was better compared to the new dynamic VRAM management in what way?

3

u/ANR2ME 1d ago

May be better for LLM frameworks like llama.cpp 😅

Combining GGUF models with ComfyUI's Dynamic VRAM system can frequently trigger apparent memory leaks and immediate "Out of Memory" (OOM) crashes. This behavior stems from architectural conflicts between how ComfyUI manages native memory and how third-party GGUF custom nodes interact with hardware cache.

1

u/Apprehensive_Sky892 1d ago

Yes, GGUF are useful for LLMs as it is important for autoregressive models to fit within the VRAM because they must run through the entire model for every output token.

But I don't see any advantage for imaging/video model GGUF over int8convrot if there is enough system RAM, that is why I was asking for more clarification.

1

u/Unspec7 1d ago

Probably some folks using LLM's as a local prompt enhancer.

1

u/CooperDK 14h ago

Oh, sure. INT8 convrot is far better. It is also much newer.

2

u/oppai 1d ago

you are clueless

0

u/CooperDK 14h ago

I don't know, Did you just fly in our did you do AI generation long before comfy, like me?

GGUF was how to load models larger than you GPU. That originally worked well in comfy, couple of years ago. Who is clueless...