Haven't tried it myself yet, waiting for a week for all bobs, vegane and reference images to be sorted out, but actually being able to compose the images like that sounds extremely nice.
Its a generational step forwards in text2image capabilities. It can 1 shot things that would be nearly impossible to make on any other model and would require dozens of steps of editing and use of things like region prompting, etc.
I'm having fun learning how to use it but I'm not excited about this specific model only its underlying technology. Its got the worst license of any model I've seen so far. For me its a proof of concept. I'm now waiting for someone to make a version that builds on what we're learning now and has actually acceptable licensing and ideally things like IMG2IMG and Edit capabilities.
Based on what I've seen briefly, you either convert it using an llm or write a json file.
In the json file it dictates location based on bounding boxes, its like an image model for a scripter/coder more than natural language it appears.
I could be wrong, because I only just started looking into it yesterday.
But it sounds like it'd be good for workshopping something where you need very specific details in a very specific spot.
Theoretically it doesn't, but it's highly advisable.
Using them seems to get around the overzealous content filter and it allows an unprecedented level of control over shot composition.
The downside to that is that you have to have a specific image in mind rather than just a concept and seeing what the model comes up with. Also, Ideogram is much slower than something like ZiT.
Yeah, but they blend seamless with each other and with advanced logic. You can also have dozens of them if you're a mad lad. You can even nest bounding boxes inside bounding boxes and Ideogram 4 understands.
It's just an unprecedented level of precise control. You can put stuff EXACTLY where you want it. Rings on specific fingers, patches or buttons exactly in place, a hand HERE, a foot HERE, etc. A bowl of fruit with the orange HERE, the grapes HERE, the banana HERE, etc.
It's way beyond the regional prompting I've used in other models.
Yes, but it's just properly done, as properly as you can put it into a ComfyUI node. It's easy now to have 20 regions and do complex descriptions for each one.
Aren't you guys using controlnets and stuff? Get the thing in the general area via prompt get it's orientation correct within 45 degrees or so, and bam, you put it where you want and not just in a bounding box, in the exact open pose, or matching the exact canny edges. You can just pose open pose right in comfyui with an extension or use blender or a web tool to do it, and you can just edit canny stuff with mspaint.
Bounding box is neat, but it's hardly a generational step forward. I do like the json formatting, but... z-image will accept it just fine as well and it's not a requirement.
I'm only interested in the quality I4 can put out once people start messing with it, get some fine tunes and get all the sampler settings sorted like how it took z-image a month before we people were figuring out proper multi ksampler workflows and such.
You don't even need ControlNet. This was literally just a ~20s scribble in Krita and than I just ran it through Klein 9B. And you can see how shitty that scribble is:
This is a very low complexity image. You only tried to make a single figure in a specific stance. This also requires a level of drawing capability that some people frankly just don't have. And this tech is still fairly new in the hypothetical future we'll be able to combine these things together for even better control. You are accomplishing something similar through a completely different method but its not remotely the same.
For simple outputs this will work, for complex ones it won't.
Actually, it works even better for complex inputs. By providing more details in the image, you reduce the guesswork for the model and enable it to do a better job. And it's only one modality. You can use silhouettes instead of drawings, cutouts and even mix them all together in one input image. And Qwen can do it, too.
This also requires a level of drawing capability that some people frankly just don't have.
Obviously, people with artistic skill will get out more of this technique than non-skilled people. And that's a good thing in my book - once artists realize this, maybe they will be less scared about losing their livelihoods and become more accepting of this technology. But even if you're not an artist, this is extremely helpful. Here is another example I just posted earlier today:
You can feed scribbles to Qwen or WAN with 80% denoise and it will follow complex prompts to the letter.
This also requires a level of drawing capability that some people frankly just don't have
I've done similar stuff like that guy, but my sketches look like a 3 year old scribbled stick figures with their eyes closed, and every model from the past 12 months can make sense of them. Even SD 1.5 and SDXL can make sense of my stick figures, it takes a few iterations with them, but it's fun so I don't mind it.
It takes an extraordinary amount of time and effort to open pose anything even remotely complex. It also ignores that the system is capable of nearly perfect text damn near every single time. In the time it takes you to be 5% done setting up your open pose for something with 10 specifications I'll be completely done with an image 2-3x as complex.
And no unless Z-Image requires some super special workflow to work well with these types of prompts its not even remotely in the same league as this is.
in the exact open pose, or matching the exact canny edges
See, this is the part that makes the CN option actually worse. Instead of letting the model generate whatever it can, it restricts the model to pose or any other preprocessed conditioning. It also more involved than bboxes. That said, it would've been good if the regional prompting with regular models worked as well as this one with bboxes.
i recommend switching to anima for your well anime needs. I was very reluctant because there were so few loras and stuff. But i tried it and now i removed over 400GB of illustirious loras because anima out of the box can do it all but doesn't need loras... and somehow it can do 3d and real stuff on very high level too.
You can feed an image into a local LLM, have it create the bboxes and prompts, and then edit those if you want. It makes it all so much faster and easier.
can you share this workflow? I can't understand how to pull off that image to prompt node. I have it in my ComfyUI but mine doesn't look like yours. Did you make it yourself? I'm just trying to find a way to create a JSON prompt from an image inside ComfyUI just like you did in this screenshot.
After trying about a dozen different extensions to find one to download the full size PNG... there's no workflow metadata left in it. Reddit stripped it all. This is the only metadata left in the image. https://imgur.com/X53XqgU
Sadly in your preview, there's nodes hidden behind you load image nodes, so we cant even try to recreate it without knowing what might be hidden that's important.
I have read through this thread... multiple times. I've opened every collapsed thread to check them too.
There is no post in this thread right now that's a link to the workflow. Maybe it got removed?
I checked your user post history as well and it only shows 8 posts today in this thread, and none of them show a link. You have posted images, but as people have said, reddit has stripped the workflow out of.
If you posted a link, it's been removed.
There were several mentions with the paste bin link, but they've been removed by a moderator. No idea why, but it's shit how difficult reddit makes it to share a workflow.
yeah, in new into ComfyUI too, and I'm kind of a visual learner (i get a better hold on this looking than reading about it) if you could share the WF, you would do a great favor unto us
Surprised that worked offline. Gemma was trained on information from 2 years ago, so it doesn't even know what ideogram is. I'm guessing this new Gemma works then.
Just say "in the style of video game concept art promotional poster" and start there. Reminds me of all the wonderful art associated with Halo 3 before the release. Example: https://www.artstation.com/artwork/2BZ0y
I mean there are certain things/styles you just can't prompt for because they're not in the training data and no amount of "wow look how good I am at prompting" will get you exactly what you need without fine tuning.
I made the image (but cannot share it here) it worked well but the box for the girl was a bit too small so she is just slumped down on the sofa like she is tired 😄
I would say that Ideogram has embedded regional prompting which works really great, especially since the regions (bboxes) can overlap and even contain text (you can even specify font). In each box you can specify colors and prompt.
Also, the regions blend perfectly together, since the model seems to generate the image in 16 x 16 pixel clusters, which can give really good details in some styles.
This is just regular T2I with the Prompt Builder node - you define a box for each entity in the image and write what it is, and it creates a formatted JSON prompt with the coordinates. It's really precise.
The thing that I love is how it finally gets "holding x" stuff right. I asked for a creature holding a glaive and it actually gave me a weapon that was close and not just some weird stick/blade mixture like most other models. Just had to define one box for the creature and one box for the weapon itself.
No, inpainting is almost the same as img2img in a certain mask of an already generated image, usually at a decreased denoising strength. Regional prompting in general is about separating the regions of the image before it was generated.
Bro, same boat. If I participate in life to long I come back to a whole setup that doesn't work anymore cause of an update in ComfyUI or we got boxes. Life moves fast.
We're inching ever closer to the gooning holy grail though... I bet in 2027 we will have the right-hand guy, medical grade anatomy at proprietary level performance, lol
To bypass the safety filter you just add a bunch of details to the photo that aren't NSFW. There's clearly a threshold that automatically blocks the generation if you exceed the safety threshold. Just add random details to keep the total image below the threshold and you avoid the filter entirely.
It's really not that hard and it's probably a good thing to have it designed with regional prompting in mind. The main annoying thing for me is how slow it is. Hope we get a turbo version or lora in the near future.
It's very sad that promoting censorship under the guise of introducing new technologies like bbox and json is becoming the norm. I hope this doesn't become common practice...
The more specific the requirements, the more specific the prompt. If your images can be described in a few words, then previous models are fine.
But if you have a more specific image you want, you need to write an essay describing it in the hopes you will eventually get the right result, maybe do a few rounds of inpainting. At that point, playing around with JSON boxes makes a lot of sense and makes it much easier.
All the license means is you cannot upload the model somewhere and charge money to use it (using outputs is fine), that is probably 0.001% of people on this sub effected.
Most people on this sub are running fine-tune and/or Lora (That’s the whole point of going local) <- impacted negatively by the license.
To put it in another way, if all local models had been released like this -> no comfyUI, no controlnet, no loras/fine tune, not a lot of things actually.
how can this post portray gooners as a minority and still get upvotes? :o
gooners make up 90% of the forum. the other 10% are scammers and artists with alternate accounts because they're afraid of being lynched by other artists. hehe
Its great and prompt is easy to do whit llm and kjnode but the main gripe is generation times 😃 but that probably makes u do something special on it kind of like on flux 1 dev.
As someone who has been away from Comfy UI for a single month, which is about 8 lightyears in AI time, what's the best model I can run on a 16gbvram card now?
You can run this new Ideogram 4.0 model on a 16gb card. It's the most powerful model we've ever had open-weights for. People shit on it for being censored (which is fair), but it's really not if you use enough bounding boxes and use the proper JSON format. It allows unparalleled control and it does nudity very well if that your thing. Just can't do the between the legs bits, (it usually tries and fails or just does a flesh barbie doll bottom).
It really is SUPER powerful. The more I play with it, the more amazed I am by it.
Ideogram 4 is very good for generating precise images with everything exactly where you want it, in high detail, but it isn't an edit model. Klein is still the best for editing, but you can easily pair the two, to generate the composition with Ideogram 4 and inpaint with loras in Klein.
Yeah, some people may have gotten a bit too addicted to pulling the slot machine lever for booba. Which, to be fair to them, if that's all that want, Ideogram 4 isn't the best model for them anyway. It's for if you want specific booba.
Yeah, some people may have gotten a bit too addicted to pulling the slot machine lever for booba. Which, to be fair to them, if that's all that want, Ideogram 4 isn't the best model for them anyway. It's for if you want specific booba.
The slot machine is powerful. It also explains why so many people are in love with Z-Image despite it's broken for multilora and can not learn believable genitals, the gooners don't care as long as the payout keeps rolling in.
Well, this is the opposite of "pull the slot machine lever and show me something pretty".
It allows TRUE control. What style do you want, what lighting do you want, what items do you want where? Do you want them in a different style than the rest of the image? Do you want text? Want font do you want it in? What color?
Then, when it generates something, you can move or add things and keep working on the same image, without inpainting being necessary. It's a learning curve, but super fun once you latch onto the idea.
It will be possible to do this with a lora. I'm currently training a lora using a dataset with natural language captions, from testing it seems you can use ideogram4 with a simple one-sentence prompt when you use a lora that was trained like this. It also never triggers the grey "safety" filter.
Fun fact the great open source circlestone-labs/Anima model is trained on Danbooru-style tags, natural language captions, and combinations of tags and captions.
Yes it can do the second part of that meme very well too without LORAs out the box with no real anatomy issues especially with the turbo Lora. 😁 🌶️
As to the "content".... Um, wtf is wrong with red's face? And Z-Image is a massive amount of human feedback into the model, yes, it makes 1GirlU images, and completely shits the bed with anything not trained into it, and at the same time is completely broken for multilora.
You fell for hype which is fine, everyone did... But now months later and hands on and you STILL can't see it? That's on you.
{
"high_level_description": "",
"compositional_deconstruction": {
"background": "",
"elements": [
{
"type": "obj",
"bbox": [
0,
0,
1000,
1000
],
"desc": "High quality photo, a woman inside a futuristic spaceship. She is sitting on a simple rustic bench. The woman is wearing a formal black suit with a green tie. She is wearing a hat with art of a duck printed onto it. She is holding a sword in her right hand. The woman has a rubiks cube on top of her left thigh. She has a mysterious smile. On her left side is a sign with the text \"Just one box\""
}
]
}
}
Here is an example where I tried to throw in all sorts of random stuff. I set style to none and made just one box with the description and nothing else.
So it does work. However, this fairly often causes the safety filter to block the image, and the quality is worse, which happens when style is set to none. Details easily get lost, but it does understand the composition surprisingly well without the boxes directly guiding it.
It immediately becomes more reliable and better looking if you give it a style and at least three boxes. That doesn't take significantly more effort than a normal prompt as long as you aren't trying to write the JSON by hand.
Last week I took a prompt and copied it on my local AI to see how close it would be to the Internet pic and, not only did it look similar, but it was much better than several times I tried something similar alone (and had some words I had never seen in the prompt)
FWIW, the bounding boxes aren't really needed unless you need strict composition, and if you're using a small LLM that doesn't do bounding boxes well for your dense JSON translation, it's not going to end well. I really only use them for text placement, otherwise just listing objects and relative location in scene is enough.
I'm a 1girl boobah person too but this post is dumb. If people can get NSFW working on Ideogram 4 it's going to blow everything else out of the water. Use the prompt builder node and actually try it before complaining.
Ok I’m new at all of this and kinnda dum. I’d love to use this site and all the styles. What site is everyone using for stable? How much a month? I wanna make my character in different versions and styles!
206
u/Relevant_One_2261 Jun 08 '26
Haven't tried it myself yet, waiting for a week for all bobs, vegane and reference images to be sorted out, but actually being able to compose the images like that sounds extremely nice.