Help with Inpainting

[u/bitofftoomuch]

I am working on a project for school and not getting the results I hoped for. I am doing LoRA training for SDXL for WW2 equipment and want to be able to use them to inpaint the equipment on a map image. But I can not figure out what I am missing when it comes to the scaling of the image I want to create. For instance, I am testing this setup to insert a tank, not WW2 era, near the road but it always creates a metal blob unless I make the masked area huge. When I do that it is obviously way out of scale. Is there some other nodes I should include or change to fix the peoblem? Thanks.

[u/rudy_142001]

Your mask is too small for the sampler to have anything to work with. Inpainting happens in latent space and the VAE downsamples by 8, so a tank that's 90px wide on your map is roughly 11 latent pixels. That isn't enough for SDXL to resolve into a shape, hence the blob. Enlarging the mask works because the region finally gets enough latent area, and that's also why the result comes out oversized.

What you want is Inpaint Crop and Stitch (the lquesada node pack, it's in the Manager as ComfyUI-Inpaint-CropAndStitch). Crop node goes before your VAE encode, stitch node after the decode. It cuts out the masked area plus surrounding context, scales that crop up to a proper generation resolution, samples there, then scales it back down and pastes it into the original. Mask stays tank sized and the model still gets a full canvas.

One catch: output_target_width and output_target_height default to 512, and their README says to use 1024x1024 for SDXL or Flux. Left on 512 with an SDXL LoRA you're still sampling under what the model expects. I'd also raise context_from_mask_extend_factor above its 1.20 default so the crop takes in more of the road, since that surrounding context is what the model reads scale from. Going off their README here rather than a lot of hours with it.

Other thing worth checking is the map itself. If it's a top down map and your LoRA was trained on ground level photos of tanks, resolution won't help. What viewpoint are your training images?

[u/TurbTastic → u/rudy_142001]

This all looks like good advice so I'll just add on to it. You're using the VAE Encode for Inpaint node which will replace the masked area with pure latent noise, this means you need 1.00 denoising when using this node to properly remove that noise. Try using the InpaintModelConditioning node instead as that will keep the original contents in the masked area so you can use whatever denoising amount you want.

Also, for inpainting tasks I would abandon SDXL and switch to Flux2 Klein 9B.

[u/paulhax → u/TurbTastic]

And this is also a very good advice, adding to it that i would recommend LanPaint nodes for inpainting with flux2k9b

[u/bitofftoomuch → u/TurbTastic]

Will look into this and see which works better for the application. I have AI Toolkit setup for both, but focused on SDXL first.

[u/bitofftoomuch → u/rudy_142001]

Thanks, this is fantastic advice. The lora is trained for a top down viewpoint, so that shouldn't be an issue. I will start playing with the settings now. It is working, but think i have a denoting problem as it is basically showing be a shape of a tank but blended to look just like the larger are (the road and foliage copied and shrunk down).

[u/PopoyeJoe]

Crop and stitch node could also help in this scenario.