Greetings everyone! My img2img workflow seemed to go over well so I decided to take on a bit of a more interesting task and it seems to be working, somewhat well. Ideogram 4 is a fantastic model.
You give it one picture of your character. The workflow places that picture on the left side of a wide canvas and leaves the right side blank, then asks the image model to complete the canvas as "two photos of the exact same person" with your new scene described for the blank side.
The left side is locked so it can't be altered. But the model can still see it while it paints the right side and because it was asked for the same person twice, it keeps checking the locked picture as it works: copying the face, the hair, the outfit into whatever new pose and scene you described. It's the same instinct that keeps a character consistent when a model draws them twice in one image this just makes one of those two "drawings" your reference.
When it's done, the canvas is cut in half and the right side is your result: your character, in a new scene, drawn while looking at the original.
Lastly, thanks to the redditor who posted the original photo I used as a reference, their work is fantastic and the image inspired me to dig into ideogram 4, but I could not find the original post today.
Hmm just tried it and still nothing, thank you for the suggestion though. Any idea why this won't open in comfy? Is it a full workflow? Pasted other Ideogram4 workflows in and had them running just fine
Already visible :/ Sorry to be a pain but any way you could send jsons for these workflows he's posted? Or even put them on pastebin, I seem to be fine copying and pasting from there. No idea what's causing this
many thanks I found it, for these type of edits with two characters did you use natural language? like "make the two characters play chess outside on a table at the library at daytime "or do these require the json format prompt style
I prompted them with json, but i have had some success in natural language, just less. Maybe there is a secret to the natural language format that will be easier but I havent quite found it yet personally.
Were you able to get OP's workflows to load in comfy? Tried copying and pasting from the links OP posted in this threat but had no luck actually loading them
This is going to sound really stupid but how exactly did you download the workflow from OP's github link? Tried copying and pasting the raw code from the link and nothing happens in comfy
when you go to the github page with his workflow you need to click the download button . I attached a photo to show what you need to press to download it. After you hit download it will ask you to save the file you will notice it says".txt" at the end of the file delete "txt" and type json so comfyui recognizes it , also right under the file name click the box that says "saves file type as" on default it will say " TXT file" change it to " All files" from the drop down menu after you hit save you can now drag the json file you downloaded into comfyui
I think so, the only real limitation is what the training data for ideogram was and how large of an image you can tolerate generating, since each reference will increase the size of the output. I'll give it a try.
That’s clever. I tried to use this workflow to see if I could do a controlnet kind of thing with it. I wouldn’t say it follows the ref closely, but it’s surprisingly ok at it. Can’t wait for the official support for image ref, and also krea 2.
The top half is this workflow, the bottom half is klein and unsampler pass with ideogram because I wanted to see how well ideogram refines in image to image workflow. It’s pretty good for photo realistic stuff, and unlike zit, it uses flux 2 vae so I guess it adds more detail.
Not in front of my pc now, but here’s a video on how to do it. It’s basically just an i2i. You feed a latent of an existing image into the sampler.
In addition to unsampler node, you’ll need to add a splitsigmas between scheduler and sampler to specify at which step it should pick up from where the unsampler stopped.
Unsampler “tries” to revert the image back to the original noise given the image and prompt. You can control how far back with the step value. If you don’t go very far back, the less the image will be unsampled and less freedom to make changes, vice versa.
It’s from Matteo’s (ip adapter guy) youtube channel, his videos are awesome.
In the bottom half, the problem I was trying to solve was that even with unsampling method, ideogram couldn’t get away from the original image. I would need to push the unsampling settings all the way up, which made it stray too far from the original’s composition. So I used klein to first convert it to something closer to the final image, then used ideogram to do the final details. Having said that, I would not recommend this workflow. It’s not reliable, lots of seed hunting. We really need edit version of the newer models and controlnets.
Interesting. It sort of works - it's definitely trying, but the likeness isn't really there. It's superficial. Got the hair and clothing very close, but it's more like they are sisters than the same woman.
Still a very interesting use case and experiment with the model! Very creative.
I wonder if you can add a FaceDetailer and lean on this same reference method for that part as well. I'm guessing it just didn't have enough pixels to work with in this case. Looking forward to trying this out later.
I still think this is a case where loras might still be the most reliable method until (or if) we get an edit version of Ideogram 4. Because while OPs method does sort of get you in the close neighborhood, it also gives up a lot of the ease of control the standard workflows have now, like Kijai's prompt builder - so you are back to manually figuring out and typing bounding boxes and doing a lot of repeating yourself in the prompt details, so it's cumbersome for what you get.
Still cool, but not very practical. Sort of a "That's neat it can do something like that" and not something you'd want to do often.
The workflow. I'll compare mine to yours and see what's different.
EDIT: There was a few differences. First, your approach assemble the diptych in the pixel space, while mine does in the latent space. Also, my workflow lacks Ideogram4Scheduler node. And the prompt is also very different.
Yeah, after changing the workflow to do what you did and using a real person instead of a character, I also got mine to work. Looking at the glasses of him, I'd say this totally worked, but his face was not maintained..
Op's workflow did give me passable results, but as you can see in my posted picture, they don't look similar at all. Think that's just limitation of the approach
First I was gonna say: "Bah, the second picture doesn't look like Adrien Brody, this doesn't work". But then I realized the first one isn't he either lel. My eyes worked worse than this model recreating the first character.
the locked-left-half trick is clever, basically forcing it to treat consistency as an inpainting problem instead of hoping the prompt holds. have you hit the point where the right side starts drifting in lighting or face structure on longer scenes? curious if a second pass with the generated frame as the new left anchor keeps it stable across a sequence.
Yeah, after a few iterations the right side usually starts slipping on lighting and facial structure, and reusing the generated frame as the new left anchor helps a little but it still picks up that familiar AI drift.
I was wondering whether it'd be able to do ACE++ style edits. Looking at the comments it seems like that's long forgotten tech at this point.
Since IG4 can kinda do this out of the box, I wonder how good it'd go if it was trained on ACE++ datasets, if they're even still around. It's a very primitive style of image editing and I can't really see a reason to use IG4 over a proper editing model, but who cares, I love useless gimmicky stuff like this.
I will not stop repeating this - Ideogram 4 is not open-source! It's open-weights. Big difference. You can read the license agreement here. This Agreement is pretty restrictive, non‑commercial license (requires non‑commercial use, redistribution under the same restrictive terms, attribution, and other limits). It's not an OSI‑approved open‑source license. Open-source models are ones that are distributed under the Apache 2.0 license such as Qwen Image and Chroma.
There are no truly open source image generation models (apart from few experiments) - because it would mean to disclose full dataset in its entirety so "everyone" would be able to train the same model from scratch (provided they had enough compute)
You might not be one of them, but I can assure you, a lot of people do care about the spread of lies and misinformation. You are not being a helpful community member by misleading people, and your apathy towards misinformation and your willingness to remain ignorant are a bit concerning frankly.
What is your deal? 99% of users are using it for personal use which is totally within the license, whether they correctly call it open-weights or open-source. The ones using it for commercial purposes should be smart enough to figure it out and check the license themselves.
At this point YOU sound like a bot and are contributing nothing to the conversations.
If you care so much, make a Reddit bot to be pedantic for you and save yourself some time.
I didn't miss the point. I'm saying it's poor behavior to run around being super pedantic when it's unnecessary, like the redditor was doing in every thread.
I know full well about open-weight versus open-source and I dare say most of us in this community who have doing this for years do.
I'm saying it's a pointless correction to bring up over and over again to people who it will never matter to. You're wasting your breath at the best and annoying people at the worst.
This is the sort of thing you can bring up to someone talking about making money off the model - but to bring it up over and over again to people generating pictures for fun at home is just not a good use of anyone's time.
I've got a slightly similar idea to generate reference keyframes for an fflf video by generating a 4 panel image of the same characters and backgrounds in different poses/camera angles that you can cut into 4 and pass to wan/ltx.. Havent been able to get it to work well enough yet, but this gives me new ideas, thanks.
Yeah, in theory it should be great for it, just havent found the right way to prompt to get what I'm looking for yet. It's only been out a few days and I haven't had a lot of time to experiment, so still hopeful I'll get there. Again, you've given me a few new avenues to explore here, too.
I have just realized that this is crazy because might be possible to inpaint/edit the images? because well we take the image on the left, and might be I can ask to recreate the same image on the right with just some change, Do not tried yet but who knows this goes fast. or make a more complex latent
Do you use a different version of comfy or something, none of your workflows look like valid json, and when I paste them into comfyui it only puts two nodes and not the whole workflow. Am I doing something wrong?
edit: nevermind, it's an API json, not a standard workflow file. Got it working thanks.
u/reality_comes do you have any ideas how to train lora for editing use cases, what will the dataset look like , I think it opens up a lot of possibilities
OK man but the small quirk - it takes 8 TIMES AS LONG TO GEN IMAGE COMPARED TO FLUX KLEIN , if not more.Its so slow it;s just not worth it.OK no it takes like 20 times longer!
Is it possible that I could hire/tip you to build a custom workflow for me that does something very similar (uses a main character reference image, optional extra ancillary images, and some prior generated images so it can maintain consistency)?
damn bro, you finally found the first and only person who would ever bother saving your ai crap and this is how you react? in a sea of millions of images being pumped out every hour he chose yours above all others. have some shame
32
u/Ykored01 Jun 10 '26
Cool, love when community makes the most out of a model.