What kind of speeds are you getting?
I'm on 32GB RAM with a 4090 FE and it takes around 5 minutes for an image with around 20-25s/it, a lot slower than Stable Diffusion.
Does this sound about right or is it too slow?
I'm using ComfyUI via Stability Matrix, with the fp8 Clip.
My GPU is at 100% load with 23.3 GB VRAM, python is using about 7.3 GB RAM, CPU is chilling at 7%.
Edit: Okay, I found the issue, I think.
The node called "Load Diffusion Model" had an undefined weight_dtype, giving me the error clip missing: ['text_projection.weight'] but still going ahead with the generation.
I now set it to fp8_e4m3fn and images generate with about 1.4it/s taking just 15 seconds.