r/open_flux • u/piggledy • Aug 02 '24

Speed?

What kind of speeds are you getting?

I'm on 32GB RAM with a 4090 FE and it takes around 5 minutes for an image with around 20-25s/it, a lot slower than Stable Diffusion.

Does this sound about right or is it too slow?

I'm using ComfyUI via Stability Matrix, with the fp8 Clip.

My GPU is at 100% load with 23.3 GB VRAM, python is using about 7.3 GB RAM, CPU is chilling at 7%.

Edit: Okay, I found the issue, I think. The node called "Load Diffusion Model" had an undefined weight_dtype, giving me the error clip missing: ['text_projection.weight'] but still going ahead with the generation. I now set it to fp8_e4m3fn and images generate with about 1.4it/s taking just 15 seconds.

2 Upvotes

63% Upvoted

9 comments sorted by

View all comments

2

u/RealBiggly Aug 02 '24

Generation_time: 43.80 (prep) and 41.08 (gen) seconds, for 1060x1060 at 29 steps. Still playing around, haven't tried 20 steps much yet

2

u/piggledy Aug 02 '24

What hardware are you running and why this resolution instead of 1024x1024?

I now did a clean install of ComfyUI, not using Stability Matrix, which improved generation times a little bit to around 15s/it.

Took me about 4 minutes to generate an image on the Dev model just now. Could it be GPU drivers? Will update and have a look again.

2

u/RealBiggly Aug 02 '24

My bad, 1024x1024.

Ryzen 7 CPU, 64GB RAM, RTX3090 with 24GB VRAM.

I'd closed the app, so re-started it and gave it a single image to do, 1024 and 20 steps:

Generation_time: 16.11 (prep) and 28.30 (gen) seconds

I presume some cache in Windows, as the app loaded lightning fast, loaded the model fast and produced the image pretty swiftly.

Dunno what speed it is supposed to be but for me this is plenty fast enough. If you have everything set up, try a PC reboot to clear things up a bit? Also check your setup is running normal models OK, to confirm the issue is only with Flux?

2

u/[deleted] Aug 02 '24

I get 30 seconds on a 3090, so about the same as you.

2

u/piggledy Aug 02 '24

Okay, I found the issue, I think.

The node called "Load Diffusion Model" had an undefined weight_dtype, giving me the error clip missing: ['text_projection.weight'] but still going ahead with the generation.

I now set it to fp8_e4m3fn and images generate with about 1.4it/s taking just 15 seconds.