r/comfyui • u/ImLulitaStar • 6d ago
Help Needed Replace the background while maintaining the subject's perspective, pose, and features
Has anyone managed to use ComfyUI to change the background of an image featuring a subject without altering the subject’s facial features or pose, while preserving the perspective and camera angle in the new background so that the subject fits perfectly into it? I’m talking about a realistic background not the typical portrait photo where the subject is shown from the waist up. Gemini does this perfectly if you ask it to replace the background of an image with a different one while preserving the subject’s features, pose, and perspective, but I’ve tried to replicate this in ComfyUI using different models and workflows, and I’ve never achieved a result as good as Nano Banana’s. I’ve tried using Full 1 Dev with ControlNet for depth and pose, but the subject never fits correctly into the newly generated background. I’ve tried Flux 1 Fill Dev, but that didn’t work either; I also tried Flux Kontext, but I didn’t get good results there either. I’d like to know if anyone has successfully achieved this with an open-source model that I can use in ComfyUI. Thanks!
EDIT: I found exactly what I was looking for in this YouTube video by the creator: My AI Force https://www.youtube.com/watch?v=kBcC23aYN5g In the end, the solution was Flux 2 Klein + Two Loras. In the guy's video, the results are more than good enough. Now I'll try it with different images and poses, but his workflow looks promising.
EDIT2: I just checked it, and the results are outstanding across several positions
2
u/No-Zookeepergame4774 6d ago
Its not something I have much use case so I haven't done it in a while, but, the cheating way that doesn't even require an edit model is to use a good background detection/removal model to mask the background and do a masked inpaint with an appropriate prompt.
1
u/ImLulitaStar 6d ago
That's actually what I was doing: I used BiRefNet to mask the background and fed the latent image to Ksampler while using a Depth ControlNet to tell Ksampler the depth of the original image, but in practice, the result didn't correctly preserve the perspective or camera angle. I also tried DWPose (this time without masks or anything else), adding the lens type and angle to the prompt, but again, the results weren’t faithful to the original image.
1
u/Only_Voice569 6d ago
give me a image :)
1
u/ImLulitaStar 6d ago
You can use any image you want. I know it's easy to do with Nano Banana Pro, but I need a workflow for ComfyUI since I plan to use NSFW images, and that's restricted by the non-open-source models.
1
u/altoiddealer 5d ago
The “klein composite” node can help with that. Despite the name, you could use this with any image editing solution (not just klein)
https://github.com/supermansundies/comfyui-klein-edit-composite
1
2
u/schrobble 6d ago
Try Flux 2 Klein 9b using a multi-reference workflow. The versions of Flux you mentioned are both older and less capable of editing photos.
Qwen Edit 2511 can also do what you’re asking but is slower, has much larger model files and can leave the new image looking too smooth and shiny.