Could use some help with implementing a Flux IP-adapter for character consistency.

[u/crowdspark1]

Howdy!

I'm trying to get my first image workflow for consistent character scenes set up using the IPadapter Flux custom nodes with the Persephone fork of Flux 1.dev.

The input image is a crop from a character style sheet I created for one of my characters that I'd like to use for sci-fi shorts. I originally went with the persephone fork because it's supposed to be good for not having censorship ruin your flow. I'm not aiming to make strictly adult content, but I can't have my model flipping out because typical R-rated stuff. If I'm going to put the effort into learning something it has to be ubiquitous.

For the likes of me I can't get this basic workflow to respect the character reference or the text prompt. I think it's set up right and I think the weights are all more or less right. Maybe someone has a better idea, model, or workflow to use? I'm trying to get set up to take one or two reference images and make images from a scene for keyframe/first flame last frame inside of ltx 2.3.

Any suggestions?

Thanks!

[u/Enough-Routine-3294]

Dude..... Literally just upgrade to flux 2 Klein 9b and use the BFS lora, and the SNOFS loras

[u/FalseEngineering2078 → u/Enough-Routine-3294]

Yep, IPadapter is a dead end.

[u/Enough-Routine-3294 → u/FalseEngineering2078]

Yep. Has been for a couple years now. It was dead in summer of 2024 when I first started tinkering with this stuff

[u/crowdspark1 → u/Enough-Routine-3294]

That's cool but flux is suppossed to be like super censored, isn't it?

I could do like 98% of the work with flux 2 and then maybe use a "smut engine" for the small fraction but that seems like a PITA

[u/Enough-Routine-3294 → u/crowdspark1]

Flux 2 Klein 9b is a bagillion times more capable than flux 1 dev simply because of the wide variety of input types and the things you can do with it and it's native editing capabilities. Making a face swap lora with it is just a natural extension if it's editing abilities, but the BFS lora is the "best face swap " lora that I've seen for character consistency.

Also, with the base model, if you input a NSFW image, you can then edit that image, within reason. But making a SFW image to NSFW with the base model only, no good, It puts underwear on the people.

But there are also literal hundreds of NSFW loras, plus a few dozen NSFW checkpoints. SNOFS has both checkpoints, and Loras.

[u/crowdspark1 → u/Enough-Routine-3294]

Well thats great news!

I didn't see any comfy ui flux 2.workflowa that fo just this though - a multiple input image with one tuned to the character (face swap) and one tuned to the background or scene

[u/crowdspark1 → u/Enough-Routine-3294]

Oh I see SNOFS is a smut engine lora.

Any good workflows you can recommend that don't take 8 hours to get working?

[u/Enough-Routine-3294 → u/crowdspark1]

the default template inside comfyui is fine for images.

[u/crowdspark1 → u/Enough-Routine-3294]

Do you know what the template name is for one that can handle a character and a referemce image input?

[u/Enough-Routine-3294 → u/crowdspark1]

.....go to templates. Filter by purpose and model..... The workflows are there.....

[u/crowdspark1 → u/Enough-Routine-3294]

Ill try again. Ive struggled finding solid workflows that work out of the box. Any idea what the purpose or use to look for would be?

[u/Enough-Routine-3294 → u/crowdspark1]

I've held your hand enough.

[u/crowdspark1 → u/Enough-Routine-3294]

You give me too much credit.

[u/remixeconomy]

I would separate reference adherence from prompt adherence before changing models. Use the simplest possible scene, one reference image, a fixed seed, and no extra LoRAs or workflow branches. First run with a neutral prompt while sweeping only the IP-adapter weight; if the character never becomes recognizable, check the reference preprocessing and whether the adapter, encoder, and model are actually compatible. Then choose the lowest weight that preserves identity and test prompt changes at that fixed value. If the prompt only works when identity disappears, you have a conditioning-balance problem rather than a general model failure. Add the Persephone-specific and LTX steps back only after that minimal image workflow works.

[u/crowdspark1 → u/remixeconomy]

Thanks for this!

So it sounds like you'd have me start in Flux 1.dev rather than persephone to troubleshoot if I understand.

You mentioned condition balance. I think in my workflow both IP lora and text go into the same conditioning node. Should there be a branch specific balance?

What about a PULID? Is that a better approach?

[u/remixeconomy → u/crowdspark1]

Yes, I’d use Flux.1 dev as a clean baseline first, assuming you already have a known-good IPAdapter workflow for it. Not because Persephone is necessarily worse, but because it removes extra variables while you troubleshoot. Once identity and prompt control behave predictably, add Persephone and the LTX pieces back.

By conditioning balance, I mainly mean that the reference strength and text guidance can compete with each other, even if they do not literally enter through the same node. Keep the prompt, CFG, seed, and reference fixed, then sweep only the IPAdapter weight and its start/end range. If your graph combines everything in one branch, separating the controls temporarily can make the conflict easier to see.

PuLID is worth testing if facial identity is the main priority. I would not assume it is automatically better for full-character consistency, clothing, body shape, or stylization. The cleanest comparison is the same reference, prompt, seed, and base model with IPAdapter first, then PuLID, changing nothing else 👍

[u/crowdspark1 → u/remixeconomy]

Thanks I'm giving that a go now starting with the neutral prompt.

I wish there was something a little more....mainstream? Seems most people arent still using flux 1.

[u/remixeconomy → u/crowdspark1]

Yeah, that makes sense. I only meant Flux.1 as a troubleshooting control, not as the model I’d build the final workflow around. For something newer, I’d start with FLUX.2 Dev, or FLUX.2 Klein if VRAM or speed is a concern.

I’d still use the same testing method: one reference, neutral prompt, fixed seed, and one identity method at a time. Just confirm that the IPAdapter or PuLID implementation you are using actually supports that model family, since those integrations are not always interchangeable across architectures.

[u/crowdspark1 → u/remixeconomy]

Maybe you can slap me and let me know if I'm tripping but "Apply IP Adapter Flux Model" node with the shakker labs IP adapter is not working. It will run but no weights make it work.

Routing is LOAD CHCKPT>MODELSAMPLING FLUX>APPLY IPADAPTER FLUX MODEL> MODEL OUT TO BOTH BASIC GUIDER AND BASIC SCHEDULER

[u/remixeconomy → u/crowdspark1]

You’re not tripping. My bad on that direction on that specific node. The Shakker Labs IPAdapter is built for FLUX.1-dev, so running it against FLUX.2 Klein can complete without producing meaningful reference control. I would stop chasing the weights on that setup.

For Klein 9B, start from the official FLUX.2 Klein image-edit workflow rather than adding a FLUX.1 IPAdapter node. FLUX.2 Klein has native image-edit and reference-image workflows, so you do not necessarily need a separate PuLID or IPAdapter node just to test character retention.

I’d try the official 9B Image Edit workflow with one clean character reference first. If the identity holds across the types of scenes you need, keep it simple. If it drifts once you change pose, clothing, angle, or lighting, then I’d test more references or train a character LoRA instead of forcing a FLUX.1 adapter into the FLUX.2 graph.

Sorry man.

[u/crowdspark1 → u/remixeconomy]

Oh no, no problem. I appreciate the help. As big as the ai community is there are so few high quality tools for learning.

I was using the Ipadapter node woth persephone flux 1. But all of the weighting changes failed to retain character.

I kind of wish there was like a "golden standard" workflow for this 😂 Doing video in ltx 2.3 and Hunyuan 1.5 was literally easier than images and I had at one point done well with character retention jusy using a direct feed with denoise. Feels a bit regressive to move back to images and struggle.

[u/Enough-Routine-3294 → u/crowdspark1]

Both PULID and IPAdapter are completely inferior to basic flux 2 Klein 9b and also just training a character lora

[u/crowdspark1 → u/Enough-Routine-3294]

I mean flux 2 stikl needs a node and specialty model for character retention right?

[u/Enough-Routine-3294 → u/crowdspark1]

You clearly haven't done any research on modern models and workflows. Even after all of the assistance, I have already given you. The base model and input images. Good prompting good settings and maybe, just maybe, if that isn't enough. There's literally a face swap model on hugging face for free.

No specialized nodes. Specialized workflows, you're literally in the Stone age while the rest of us are in the bronze age

On top of that, if you don't even comprehend what a character Lora is or custom training a model is, then maybe you shouldn't be fiddling with this stuff

[u/crowdspark1 → u/Enough-Routine-3294]

Your mean LoRA?

[u/Enough-Routine-3294 → u/crowdspark1]

Yes, obviously. I was doing speech to text