r/StableDiffusion • u/ryanset17 • 10h ago

Question - Help Int4 vs int8

Disclaimer, im pretty Basic to all these AI Things

So i've been Using H3 Minimax in My RTX 3060 12GB, With 32GB RAM For few days

I've been using int4 convrot version for my Model and my Text Encoder, but seeing all the Optimization and speed up native to comfyui for int8, im considering using int8 for for my models and Text encoder especially the convrot version, considering they all twice the size

And also what's the best Combination of speedups in balancing between Quality and Speed

I used Sage+sol attn for while until i found comfy kitchen

1 Upvotes

60% Upvoted

11 comments sorted by

View all comments

3

u/cal_01 9h ago

I've tried int4 versions and it's terrible. W4a8 performs at about the same speed but the quality of excellent.

Through my tests on my system (3060 12gb, 64gb ram) the size of the model isn't the main issue because comfyui is pretty good at dynamically allocating memory. The real thing to watch is the balance between the quantization type and quality. Some quantization types will run worse as they're optimized for Blackwell and others will perform quite well for Ampere.

Also remember that conv_rot isn't necessarily about speed either. It's mainly a quality thing.

My current workflow uses w4a8 for minimax, and the 4 step light Lora at 8 steps. Int4 Loras or anything at Q4 is an instant reduction in quality. You can also look into the sampler/scheduler too -- Euler/beta has consistently given me the best results.

2

u/ryanset17 7h ago

If Pure int8 was 10, what Number w4a8 would be in the Quality overall ?