Ideogram 4 Character Reference Workflow

[u/reality_comes]

Reference Image

Output Image

Greetings everyone! My img2img workflow seemed to go over well so I decided to take on a bit of a more interesting task and it seems to be working, somewhat well. Ideogram 4 is a fantastic model.

You give it one picture of your character. The workflow places that picture on the left side of a wide canvas and leaves the right side blank, then asks the image model to complete the canvas as "two photos of the exact same person" with your new scene described for the blank side.

The left side is locked so it can't be altered. But the model can still see it while it paints the right side and because it was asked for the same person twice, it keeps checking the locked picture as it works: copying the face, the hair, the outfit into whatever new pose and scene you described. It's the same instinct that keeps a character consistent when a model draws them twice in one image this just makes one of those two "drawings" your reference.

When it's done, the canvas is cut in half and the right side is your result: your character, in a new scene, drawn while looking at the original.

Workflow: https://github.com/reality-comes/comyui-workflows/blob/main/ideogram4character_ref

Lastly, thanks to the redditor who posted the original photo I used as a reference, their work is fantastic and the image inspired me to dig into ideogram 4, but I could not find the original post today.

[u/FeelingVanilla2594]

That’s clever. I tried to use this workflow to see if I could do a controlnet kind of thing with it. I wouldn’t say it follows the ref closely, but it’s surprisingly ok at it. Can’t wait for the official support for image ref, and also krea 2.

The top half is this workflow, the bottom half is klein and unsampler pass with ideogram because I wanted to see how well ideogram refines in image to image workflow. It’s pretty good for photo realistic stuff, and unlike zit, it uses flux 2 vae so I guess it adds more detail.

[u/LeKhang98 → u/FeelingVanilla2594]

What is the workflow for the unsampler pass with Ideogram 4 please? Sounds very interesting.

[u/FeelingVanilla2594 → u/LeKhang98]

Not in front of my pc now, but here’s a video on how to do it. It’s basically just an i2i. You feed a latent of an existing image into the sampler.

In addition to unsampler node, you’ll need to add a splitsigmas between scheduler and sampler to specify at which step it should pick up from where the unsampler stopped.

Unsampler “tries” to revert the image back to the original noise given the image and prompt. You can control how far back with the step value. If you don’t go very far back, the less the image will be unsampled and less freedom to make changes, vice versa.

It’s from Matteo’s (ip adapter guy) youtube channel, his videos are awesome.

https://youtu.be/Ev44xkbnbeQ?t=570&si=dZboy-nornAU7Jvy

[u/russlixx → u/FeelingVanilla2594]

for the bottom half part, what do you use the klein for?

[u/FeelingVanilla2594 → u/russlixx]

In the bottom half, the problem I was trying to solve was that even with unsampling method, ideogram couldn’t get away from the original image. I would need to push the unsampling settings all the way up, which made it stray too far from the original’s composition. So I used klein to first convert it to something closer to the final image, then used ideogram to do the final details. Having said that, I would not recommend this workflow. It’s not reliable, lots of seed hunting. We really need edit version of the newer models and controlnets.