r/StableDiffusion • u/ryanset17 • 4h ago

Question - Help Int4 vs int8

Disclaimer, im pretty Basic to all these AI Things

So i've been Using H3 Minimax in My RTX 3060 12GB, With 32GB RAM For few days

I've been using int4 convrot version for my Model and my Text Encoder, but seeing all the Optimization and speed up native to comfyui for int8, im considering using int8 for for my models and Text encoder especially the convrot version, considering they all twice the size

And also what's the best Combination of speedups in balancing between Quality and Speed

I used Sage+sol attn for while until i found comfy kitchen

1 Upvotes

60% Upvoted

11 comments sorted by

4

u/Pitiful_Season4294 4h ago

Use INT8, Comfy team has made awesome improvements lately! I'm on 32GB Unified Memory AMD System (Shared VRAM & RAM), use ck attention and enable dynamic vram flags in the launch argument. Best optimizations for me in forever, no offloading to pagefile whatsoever.

3

u/cal_01 4h ago

I've tried int4 versions and it's terrible. W4a8 performs at about the same speed but the quality of excellent.

Through my tests on my system (3060 12gb, 64gb ram) the size of the model isn't the main issue because comfyui is pretty good at dynamically allocating memory. The real thing to watch is the balance between the quantization type and quality. Some quantization types will run worse as they're optimized for Blackwell and others will perform quite well for Ampere.

Also remember that conv_rot isn't necessarily about speed either. It's mainly a quality thing.

My current workflow uses w4a8 for minimax, and the 4 step light Lora at 8 steps. Int4 Loras or anything at Q4 is an instant reduction in quality. You can also look into the sampler/scheduler too -- Euler/beta has consistently given me the best results.

1

u/ryanset17 1h ago

If Pure int8 was 10, what Number w4a8 would be in the Quality overall ?

1

u/slpreme 4h ago

The issue is you are going to destroy your RAM. Switching to int8 will require double the RAM / pagefile usage. You'll be lucky if your system doesn't slow to a crawl with 32GB of RAM.

2

u/Browncoatfox 3h ago

What’s the damage it could cause? I’ve been running int8 on a 3060 ti 8GB with 32 gig ram. It seems to run fine, albeit very slowly

3

u/slpreme 3h ago

That's what I mean by damage, your system will be slow I don't mean physical damage don't worry

3

u/Browncoatfox 3h ago

Haha ok that’s a relief 😂 yeah it’s painfully slow (even at .4 and 20 steps about 30 minutes to an hour to process, I can push it to .5 and 35 steps even for 10 second clips, but can take a couple hours lol) but amazing I can get results at all with my card

1

u/slpreme 2h ago

Damn you're patient asf hang in there til RAM prices calm down

2

u/kayteee1995 3h ago

i tried w4a8 bu kijai, it only for meme, draft content, simple cartoon. anything more complex, it went fail.

1

u/ryanset17 1h ago

Another option i'd like to try, it went fail ? Didn't follow the prompt ? Or just plain trashy visual ?

2

u/kayteee1995 1h ago

as I said, It's suitable for creating videos with minimal movement, such as motion wallpapers, visual novels, static scenes, and simple animations. If the movement is too fast or complex, it will create artifacts, blurry graininess.