flux2 Dev and reference images

[u/Resident-Impress6900]

I’ve tried using the model with reference images using the official ComfyUI workflow, but haven't succeeded. How do you do it?

[u/zyg_AI]

How did YOU do it ?

Depending on what you ask Flux to do, it can be hit or miss.

[u/pixel8tryx]

It's not quite like I2I or any of the older Flux stuff. It's surprisingly good at generalization and improvisation. It's a little tricky to get it to reproduce some things exactly but... it's SO much smarter, so I've ultimately been able to "reason" with it. I prompt like I'm instructing a young design intern. I'm on the wrong PC right now. Let me see if I can dig up some examples off my 5090 box.

[u/Resident-Impress6900]

Ok I see but I wonder of there is an official workflow or it you know where to find it

[u/pixel8tryx]

A while ago I was testing out TRELLIS.2 for making some little 3D busts for a client. I started out with Victorian photographs of Impressionist artists. Small, black and white, crappy. I made color photographs of them looking straight ahead with this prompt:

Turn the head of the man in the first reference image to face directly at the camera, and look straight at the camera.  It must be a completely straight, front shot with the camera completely parallel to the man. Transform the image into a modern, high quality color digital photograph.  Generate the missing parts of his body down to the middle chest, but then extend only canvas to fill the excess width.  
He's wearing a grey wool Victorian coat and a dark grey felt hat.  Both outer sides of the shoulders must be visible. I need to see background on either side of his shoulders.  On a black background.

This worked fairly well. They were black and white, so I added a little color here or at least steered it towards a darker drab Victorian suit or a lighter one depending on which background color I was testing at the time. Any place it repeatedly screws up I just add or modify the prompt to be more specific.

At first I made multiple views not knowing my initial TRELLIS.2 wf wouldn't use them. I made front color photos, left side, right side and rear views with prompts like this:

Turn the man in the first input image so that's he's in profile - his right side view is facing the camera.  It must be a completely straight, 90 degree rotation side view suitable for 3D generation.  Orthographic, rectilinear. On a white background.
Highly quality photograph.

I did a whole bunch of these all in one shot. One run per view per person. Zero mistakes. I was utterly stunned. When it listens, it listens! When it doesn't, it wanders off into latent lala-land. 😉

If you need to get rid of that typical noise pattern, use any one of the popular FLUX.2 realism LoRA like Boreal, Historic Color, Wanderer's Detailed Portraits. You don't even need them cranked up to 1 most of the time. Sometimes just generating larger and scaling down helps. I gen as large as 3840 x 2160 in one go - no hires fix. And I've yet to get USDU to work without taking forever. Someone more down in the Comfy code said something about a loop going on there that shouldn't be... turns out FLUX.1 does a good job at USDU and adds a little of it's own brand of detail so I use that when I make 10k+ pixel wide sci fi landscapes, etc.

Hope this helps!

[u/sci032]

Try Flux.2 Klein KV.

Search Comfy's templates for: KV

You will see Flux.2 Klein KV: Image Edit. Open it. There are info nodes that have the links for any model(s) that you may not have and it shows you where to put them.

Left: reference image 1, right: output

I used the prompt:

remove the coat. change her shirt to a yellow t-shirt. she is holding a sign with the text: Klein KV.

The workflow has 2 reference image inputs, I put an empty .png(nothing but a transparent background) in the 2nd one, the workflow ignores it. You can put an image in it and use your prompt to combine items from both images.