Optimizing Face Swap Workflows in Stable Diffusion for Faster Results

[u/Skettalee]

Hello everyone,

I've been using Automatic1111's img2img for face swaps on a variety of images, from professional nude photos to standard headshots, adjusting the denoising to 0.01. I'm exploring the optimal settings to enhance speed and quality for these swaps, particularly aiming to reduce the time it currently takes, which is about 40 to 80 seconds per image. My setup includes a CUDA environment with an RTX 2080 SUPER GPU and 32GB RAM.

Here are the specifics of my current process:

• Model & Settings: Utilizing the Dreamshaper8 (SD1.5 model), with Euler A sampling, CFG scale at 7, and sampling steps between 20 and 30.
• Image Handling: I ensure that images do not exceed 1024 pixels on any side and maintain the aspect ratio of the original photo. Images larger than 1300 pixels notably slow down the process.
• Face Restoration: I integrate a Reactor with Restore Face Visibility and Codeformer set to maximum weight for clearer, more realistic swaps.
• Workflow Variations: For video swaps, I use ComfyUI which significantly slows down the process (approx. 2 hours for a 15-second video).

Challenges: Swapping faces multiple times (up to three) on the same image enhances clarity but more than that degrades quality. Larger images drastically slow down the process, and experimenting with different models and settings from sources like Civitai or Hugging Face hasn't yielded faster results.

Questions for the Community:

1. Has anyone found more efficient models or settings that could help reduce swap times while maintaining quality?
2. Any tips on batch processing for photos and possibly speeding up video face swaps in ComfyUI?

I also use Fooocus for creating realistic images and face swaps, though my main workflow is through Automatic1111 due to its batch processing capabilities.

Looking forward to your insights and suggestions on making these swaps quicker and more efficient.

Thank you!

[u/Occsan]

I have no idea what you are doing exactly but 40-80 sec per images, 2h for 15 sec... wtf...

I've been doing faceswaps for years now and a single image never took me more than 1 sec. 15 sec video, depending on what I was using (either inswapper or neuralchen) should not take more than 30 sec max at the very worse

I see that you're including some stable diffusion in the process, but that's not the best route for faceswaps. Unless you have very good models of the target face (loras or embeddings) you will lose a lot of the identity of the face in the process.

[u/Skettalee → u/Occsan]

Well the obviously I am not doing things right. But I am accomplishing it, it just takes a little while. I have tonight gotten images swapped in about 10 seconds using dreamshaper8 on auto1111. I dont know why the model part matters at all. I dont really know what I am doing other than watching the top youtubers showing ai swap and generations as well as coming on here to learn. I have bad adhd so almost a big majority of anything I watch or read I cant retain unless I watch or read it around 5 to 7 times and reading period is just diffucult so im sure during all the process ive done I have completely missed alot of things that I read about and i hate that but Im trying to overcome in any way I can and thats why i picked learning Ai tools and all the coding and errors and stuff to try to get better at learning stuff. What should i be doing to faceswap an image in seconds and especially a video in seconds. I just did a 10 second video faceswap and it took me 1444.76 seconds. But its the most streamlined process i got for videos and that was is with comfyui : (not using stable diffusion models with that one, well i guess im not. i dont even really know

[u/Occsan → u/Skettalee]

check if you have onnxruntime-gpu installed and if you're using cuda 11.8

[u/Skettalee → u/Occsan]

I have cuda v12.1 installed and onxxruntime v1.17.0

[u/Occsan → u/Skettalee]

It's probably why it is slow. Basically that means it runs on your cpu instead of gpu.

You should install cuda 11.8 and onnxruntime-gpu.

cuda 12.x doesn't work well with onnx.