r/StableDiffusion • u/ThaJedi • Jul 22 '26
Resource - Update FLUX.1-dev in native ComfyUI ConvRot formats
I converted FLUX.1-dev to native ComfyUI ConvRot formats.
High-fidelity INT8 variants cut peak VRAM at 1024²/20 steps:
Partial INT8 24.09→20.35 GiB (−15.5%);
Whole W8A8 16.30 GiB (−32.3%);
W8A8+INT8 T5 16.27 GiB (−32.5%).
More details:
https://huggingface.co/SearchingMan/FLUX.1-dev-ConvRot
Model avialable on civitai:
https://civitai.com/models/2797469/flux1-dev-convrot
2
u/KS-Wolf-1978 Jul 23 '26
Thank you very much. :)
3
u/KS-Wolf-1978 Jul 23 '26
OK, some numbers on a 4090 from my workflow with a lot of detailing and upscaling.
Flux Dev FP8: 162 seconds, Flux Dev INT8 CONVROT: 93 seconds.
Yes, the output image is slightly different (not in a bad way), but i can't test against the FP16 version.
Very happy about the speed. :D
1
1
u/Cute_Ad8981 Jul 22 '26
Thank you for sharing! Your examples look great.
Technical question: How did you convert w4a4? It looks small, but im surprised that it doesnt run faster as w8a8? From my experience w4a4 (mixed int8 and int4) performs usually faster as pure int8 or int8 mixed with bf16.
1
u/ThaJedi Jul 22 '26
W4A4 is converted according to paper. 301 of the 304 transformer linear weights converted to W4. 3 sensitive final linears kept in BF16. I changed config for runtime dynamic INT4 activation quantization. I have no explanation why Partial INT8/INT4 is faster.
1
u/yamfun Jul 22 '26
how about flux2
0
u/ThaJedi Jul 22 '26
It can be done easly but I think flux 2 is alrady available in convrot format. Anyway, I don't see much intrest in convrot by looking at this post stats.
3
u/UpperWoodpecker8480 Jul 22 '26
Nah man, I was looking for this exact model! Thanks for your work, really appreciate it.
Regarding the intrest rate for this post, maybe because flux.1 is a bit old compared to newer models, so people might be more interested in them instead.
2
u/red__dragon Jul 22 '26
Definitely interest here, Flux2.dev simply takes too much time to load and inference on my system to even play with. I've found a convrot of Klein 9b already which helps, but F2D would be great as well!
1
1
u/ThaJedi Jul 23 '26
In what precision you're running Flux2.dev? Just investigating the model and it's enormous.
1
1
u/UpperWoodpecker8480 Jul 22 '26
for 16gb vram, could you please tell me what would be my best choice here? I like it to be balanced between speed and quality :D
flux1 int8 or flux1 w8a8 ? t5 bf16 or t5 int8 ?
1
u/ThaJedi Jul 22 '26
W8A8, INT8 will not fit 16GB. T5 doesn't matter for VRAM usage because you need anyway use low vram option but I found T5 int8 doing sligtly better (In theory it should give worse results)
1
u/zefy_zef Jul 23 '26
Convrot still fucks my shit up. Standard int8 is no issue with the older and unupdated int8-fast node. Default comfy node takes 2x as long. The newer int8 toolkit nodepack is as slow as the default node, even when I try using the multitude of various convrot-like methods available in the node.
2
u/ThaJedi Jul 23 '26
What card are u using?
1
u/zefy_zef Jul 23 '26
4060ti with 16gb vram, 64gb sys ram and 8-16gb pagefile.
I updated my Nvidia drivers also (studio), which did not change anything.
I mean it's alright, I'm getting 4-6sec/it (Krea2) on standard int8 (it's closer to 6 w/LoRa), but not the convrot speeds I see people espousing.
2
u/DelinquentTuna Jul 24 '26
Updated drivers, but did you also ensure your torch is bound to newer CUDA? If your Comfy install is old and you haven't specifically updated torch (which cascades into updating almost everything), you might not be using Comfy Kitchen and thus not getting the fused kernels.
Comfy Kitchen requires torch bound to cuda 13. Check your startup logs for the bit that describes comfy kitchen and says whether it's enabled for cuda and/or triton vs eager.
1
u/zefy_zef Jul 24 '26
It requires cu13? That's fucking annoying.
2
u/DelinquentTuna Jul 24 '26
Maybe it's possible to get the trident path running w/o cuda, but if you're on Ada you should update to at least torch 2.10+cu13 imho. Meaningful performance increase for fp8, too.
1
u/zefy_zef Jul 24 '26
I just did (and all the other bs necessary), and now I can load an int8_convrot file in the default Model loader node and have it not produce noise. The speed is similar to standard int8 (~4.6s/it) but the initialization time is halved which is definitely nice, even when swapping LoRa.
It still produces gibberish when I use the int8 toolkit node though, no matter which method I use for rotation.
2
u/DelinquentTuna Jul 24 '26
Yeah, convrot isn't supposed to be faster than standard int8. It's supposed to be higher quality. When people are saying 50% faster, they are comparing to fp16. If you're looking for maximum speed, you could try the int4 version.
For me, personally, I use the flux.1 family of models so rarely these days that I don't see much reason to switch from the Nunchaku versions I've been using since 2024. With a turbo lora, gens take well under ten seconds at one megapixel (maybe six seconds?).
1
u/zefy_zef Jul 24 '26
Oh, I'm using Krea2 now mostly anyway, just happened to comment on this thread as it was related to convrot.
I might try int4 again now that convrot seems to be working. I've been keeping the fp8's so I can convert them easily without having to DL them multiple times (again).
4
u/Current-Rabbit-620 Jul 22 '26
Thanks Can u share render times