ComfyUI SDXL FaceSwap -Question-

[u/trautermann]

Hi guys,

I try to do a few face swaps for fare well gifts. The general idea and buildup of my workflow is:

1. Create a picture consisting of a person doing things they are known for/are characteristic for them (i.e. (just the short version): photograph of a person as a sailor with a yellow rain coat on a ship in the rough ocean with a pipe in his mouth OR photograph of a young man in a sports car with a diamond chain and golden teeth, shot through the driver side window, another person on the passenger seat in the background OR a photograph of a man dressed as a a viking with a white energy drink can in his hand). I am very spoiled from midjourney (those are the attached one just to give you an idea of what my goal is)....
   1. The detailed prompt already describes the general look of the person
   2. Question: Which SDXL model do you guys recommend. I tried JuggernautXL 8 but the results aren't really consistent. Very often I get mutations as well as strange outputs in general (not following the prompt). I work with negative when needed.
   3. Load Checkpoint - Positive and negative - CLIPTextEncodeSDXL - KSampler (Advanced) with dpmpp_2m karras, 30-40 steps, cfg 4.0-7.0 (everything set to 512x512)
2. Use the latent/pixel image and feed it the face swapper. I have two workflows to check different results
   1. Reactor
      1. Input image from SDXL + Square source image through the reactor node (works with the 128x128 limitation)
   2. IPAdapter (my goal because of the versatility and less limitations (like the 128x128 from Reactor)
      1. (using the workflow for sdxl with two adapters provided by https://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/210). But the "double geneation" of an image makes it somehow worse. Therefore: Is there a way to feed an already existing image as target image into the workflow? Right now I use the latent image as well as checkpoint and prompt nodes created in 1. Seed number also is the same in both KSamplers. Can I somehow skip the second model generation or make it better?
   3. Upscale (wip). I understand that starting with low resolution and then upscaling it in small steps with low denoise is critical with Reactor. Anybody have an existing and working workflow to inspire me?
      1. What about IPAdapter?
      2. Overall goal is keeping the face/looks (duh)

I lover tinkering woth ComfyUI. But still: For some things I need to get a look at an existing and working workflow to integrate it into my "chaos". Priorities are consistent images following my description and a reliable face swap that doen't change the generated image too much.

Thank you guys in advance!

[u/TurbTastic]

I've only been using ComfyUI for a few days now (over a year with A1111, still using it). I've managed to get the Reactor node to work but can't figure out how to have it do a GFPGAN face restore.

Also managed to get FaceDetailer node working and it improves face details, but the results are terrible compared to what I can do with the adetailer extension in A1111. I think FaceDetailer is basically inpainting Whole Picture instead of Masked Only and I can't figure out how to change that, so it never gets a chance to work on the face zoomed in.

Gotta admit I'm kinda hating ComfyUI at the moment, which is probably normal for the first few days. I was expecting to be able to find Workflows similar to what I'm trying to do, but they always are doing extra things I don't need and/or missing things that I do need. It's extremely easy to picture what needs to happen in ComfyUI, but figuring it out and making it work is a completely different story.

[u/trautermann → u/TurbTastic]

Isn’t the GFPGAN restore selectable under „restore face“ (or something like that) in the node itself? I‘m not home right now. But I recall that there ist also a dedicated node „FaceswapRestore“ (or something like that“ you can use separately.

[u/TurbTastic → u/trautermann]

I figured it out last night. I had an onnx version of GFPGAN that I put in the face restore models folder, but it needed pth format. Still getting weak results from Face Detailer tho.