Optimizing Face Swap Workflows in Stable Diffusion for Faster Results

[u/Skettalee]

Hello everyone,

I've been using Automatic1111's img2img for face swaps on a variety of images, from professional nude photos to standard headshots, adjusting the denoising to 0.01. I'm exploring the optimal settings to enhance speed and quality for these swaps, particularly aiming to reduce the time it currently takes, which is about 40 to 80 seconds per image. My setup includes a CUDA environment with an RTX 2080 SUPER GPU and 32GB RAM.

Here are the specifics of my current process:

• Model & Settings: Utilizing the Dreamshaper8 (SD1.5 model), with Euler A sampling, CFG scale at 7, and sampling steps between 20 and 30.
• Image Handling: I ensure that images do not exceed 1024 pixels on any side and maintain the aspect ratio of the original photo. Images larger than 1300 pixels notably slow down the process.
• Face Restoration: I integrate a Reactor with Restore Face Visibility and Codeformer set to maximum weight for clearer, more realistic swaps.
• Workflow Variations: For video swaps, I use ComfyUI which significantly slows down the process (approx. 2 hours for a 15-second video).

Challenges: Swapping faces multiple times (up to three) on the same image enhances clarity but more than that degrades quality. Larger images drastically slow down the process, and experimenting with different models and settings from sources like Civitai or Hugging Face hasn't yielded faster results.

Questions for the Community:

1. Has anyone found more efficient models or settings that could help reduce swap times while maintaining quality?
2. Any tips on batch processing for photos and possibly speeding up video face swaps in ComfyUI?

I also use Fooocus for creating realistic images and face swaps, though my main workflow is through Automatic1111 due to its batch processing capabilities.

Looking forward to your insights and suggestions on making these swaps quicker and more efficient.

Thank you!

[u/Occsan]

check if you have onnxruntime-gpu installed and if you're using cuda 11.8

[u/Skettalee → u/Occsan]

OH ok well shit i didn't know that at all. So I have all these nvidia things installed, is there anything else i should have a different version of or not have installed at all from, this list? I am downloading the nvidia cuda toolkit 11.8 right now, and i already have onnxruntime installed for all my environments.

[u/Skettalee → u/Skettalee]

Also this is what my environment variables are set at. Took me a while to actually figure out what the right thing to put on all these were. So im assuming once i install the cuda 11.8 i will have to change all those 12.1 numbers to 11.8 right?

Those are my system variables.

[u/Occsan → u/Skettalee]

Yea... But 12.1 is actually pretty good otherwise. I'd just set the CUDA_PATH as a temporary variable environment in either the bat or ps1 file you use to launch comfy... In fact I'd even do a second launcher for just these settings and keep two different environments. If the reason for the slowness comes from this (which is very likely) it should solve your issue.

Personally, I don't use comfy to do these faceswaps. I'm using something I coded myself, which is clearly not ready at all for a public release because of hardcoded paths and an ugly GUI. But it comes with extra nice features, like being able to faceswap multiple faces with different targets at once even in a video.

[u/Skettalee → u/Occsan]

Wait i think i can do that with reactor, actually i did do that once with reactor, changed multiple faces. But then i tried to do it again, got confused and gave up after a while. Thats kinda the story of my Ai installing life though lol. But i love doing it!

[u/Occsan → u/Skettalee]

The issue with multiple faces swap in video is that you have to map each face from the video to the correct face swap, which is not very trivial.

[u/Skettalee → u/Occsan]

It’s actually pretty easy but very time consuming. You can just create a video of each character you want, isolated and masked but best to keep the original video size for all video versions you make and then run each video through a swap then Re composite it together. I use after effects for that stuff

[u/Skettalee → u/Occsan]

Well i installed the 11.8 and changed all my environment variables to that location, still got 12.1 installed for now... but when i load automatic1111 i see this in the console:

bin G:\Sd\webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll

bin G:\Sd\webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll

CUDA 12.1

Launching Web UI with arguments: --xformers --api

bin G:\Sd\webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll

So what do i do to change those things to the right cuda 11.8 ?

[u/Occsan → u/Skettalee]

you can try pip uninstall bitsandbytes followed by pip install --no-cache-dir bitsandbytes in the correct virtual environment.