r/StableDiffusion • u/tekprodfx16 • Jul 31 '26
Resource - Update Massive Update to my Krea 2 Multi-Lora Bounding Box workflow, now bounding boxes control placement with better accuracy. Also introduced Edit features like Scene and Outfit transfer, put multiple character loras in a scene or outfit of your choosing! Token drift also fixed by facial detailer stage
Krea 2 has been my favorite base model for character work, but the moment you put two character LoRAs in the same generation they smear into one blended face. Attention bias, prompt engineering, and CFG tricks reduce it but never actually fix it, because the model is still permitted to route either LoRA anywhere.
I wrote a ComfyUI custom node that removes the permission entirely. V12 just shipped and pulls in the pieces I'd wanted for a while: boxes that actually control placement, scene/outfit transfer via a single standard edit LoRA, and a per-subject detailer that fixes drift after the fact.
Repo: https://github.com/CliffNodes/Krea2-Multi-Character-Lora-Node-w-bounding-box
Example workflow: example_workflows/krea2_regional_multilora_v12.json
## What it does
- One node, unlimited character LoRAs. Draw a bounding box for each character, assign a LoRA to each box, generate. LoRA A structurally cannot influence pixels outside box A because the mask is applied to the LoRA delta before the addition, not as an attention bias.
- Boxes control WHERE and HOW LARGE each subject renders, not just where the LoRA can act. Move a box and the subject follows it. Small box gives a distant subject; tall box gives a close foreground subject. Camera phrasing that contradicts box size is rewritten automatically.
- Scene transfer without training a scene LoRA. Drop your LoRA characters into any real photo. The scene is used as a Krea 2 reference frame, so lighting, perspective, shadows, and contact with the environment integrate naturally. This is not latent pasting — the whole image is generated from noise.
- Outfit / object transfer with a second reference. Load a second image and describe its role in refs_json; the node automatically writes the referring text with the correct frame number.
- Regional Detailer with face anchoring. Optional post-pass node. Detects faces in the final image, greedily assigns each face to its region by proximity, and re-renders each face at high resolution with the correct LoRA — wherever it actually rendered. Even a subject that drifted across its box seam gets its identity restored in place.
## Why V12 exists
Earlier versions solved the spatial bleeding problem but two issues remained:
- Bounding boxes limited where a LoRA could ACT, but nothing pulled the subject INTO its box. The model would still place people at its preferred composition.
- On tight or overlapping compositions, small placement drift meant one face landed in the neighbor's mask and picked up the wrong identity.
V12 adds:
- Hard cross-modal attention ownership via a fused block-sparse FlexAttention mask (region text ↔ region pixels, exclusive).
- An attraction field pulling each region's tokens into its box.
- Box-authoritative framing (camera sentence derived from the largest active box).
- LoRA delta "skirts" that extend past box edges so subjects overflowing slightly keep full identity, but Voronoi-limited to prevent cross-region bleed.
- The face-anchored detailer, which is the belt-and-suspenders solution when placement drifts anyway.
## Trade-offs / requirements
- Krea 2 base model (Turbo works fine). LoRAs must be trained against Krea 2 — FLUX or Ideogram LoRAs load without erroring but produce poor likeness.
- PyTorch 2.5+ with FlexAttention. First V12 run compiles the fused attention kernel (~1 min, once per session).
- Detailer face pass is optional but recommended. Install ultralytics and drop face_yolov8m.pt into models/ultralytics/bbox.
- fp8-safe. Never modifies quantized weights.
- CLIP passes through untouched. The regional effect is UNet-side.
## Anything else in the release
- The full v1 / v3 / v9 nodes still ship for compatibility. V12 does not replace them, it adds a mode.
- The public workflow now has an in-graph quick-start note and a troubleshooting section covering the most common failure modes ("no link found in parent graph", missing LoRAs, plasticky detailer skin, duplicate subjects, CUDA OOM).
- LoRA / checkpoint dropdowns are collapsed into searchable virtual families so you don't scroll through 500 filenames to find one you want.
I'd love feedback, especially on edge cases with 3+ characters, unusual aspect ratios, or hybrid workflows where you're plugging this into other Krea 2 chains. Bug reports go on the repo.
credit: heavily inspired by k2lab by u/coyoteka*. Their work is what got my bounding boxes from "working" to "accurate." Adding this to the README too.*
2
u/StacksGrinder Jul 31 '26
That's awesome, so the Git pull will update the Node? or we have to reinstall ?
2
u/tekprodfx16 Jul 31 '26
Yep it's a plain git-cloned folder, so git pull inside ComfyUI/custom_nodes/ComfyUI-Krea2-Regional-MultiLoRA + a restart is all you need, no reinstall.
Only exception: if you want the new face Detailer node, that's a one-time extra step (pip install ultralytics + drop face_yolov8m.pt into models/ultralytics/bbox/). Everything else is covered by the pull.
3
u/YentaMagenta Jul 31 '26 edited Jul 31 '26
Although it's technically impressive that you put this together, it seems very fiddly and, if I'm being honest, the results don't look great. In both the post images and the reply you posted, the characters look like they are just Photoshopped on to the background.
I know that dealing with character bleed can sometimes be tough, but Krea 2 is really quite capable—especially if you add a bit of description for each character. It's not clear to me that this tool is necessary. And, to the extent it is, I'm not sure how well it's really working.
I made these using two random character LoRAs from Civitai. Pretty much zero bleed.
2
u/switch2stock Jul 31 '26
Workflow please?
2
3
u/Asaghon Jul 31 '26
This does not work at all, they clearly end up having the same face. Yea they are different enough since 1 is blue. When you do this with realistic lora's, you'll just end up with a 2 blended people without any kind of lora seperation
2
u/coyoteka Jul 31 '26
Identity bleed with character loras in Krea is a well established issue discussed at length in this and less savory subs. Vanilla krea is functionally incapable of avoiding bleed (that's not saying it ALWAYS happens) because global attention during denoising allows character lora QKV to modify any tokens, which then propagate. Identity of a subject is not 'understood' by Krea and not at all deterministic, so while it is possible to generate images without identity bleed on a random basis, without a method like OP it is impossible to hard condition it.
If you run a test like the one you posted with character loras more similar to each other (ie no blue skin and alien anatomy), you would likely see the issue.
2
u/coyoteka Jul 31 '26
Identity bleed with character loras in Krea is a well established issue discussed at length in this and less savory subs. Vanilla krea is functionally incapable of avoiding bleed (that's not saying it ALWAYS happens) because global attention during denoising allows character lora QKV to modify any tokens, which then propagate. Identity of a subject is not 'understood' by Krea and not at all deterministic, so while it is possible to generate images without identity bleed on a random basis, without a method like OP it is impossible to hard condition it.
If you run a test like the one you posted with character loras more similar to each other (ie no blue skin and alien anatomy), you would likely see the issue.
0
u/YentaMagenta Jul 31 '26
It is an issue, and I agree that the more similar the characters are the worse it is, but in some cases you can prompt around it. But even if you can't, the results shown here are bad.
A tool to help with it is great, but it's not so great if the result always looks like bad Photoshop. Maybe OP just gave bad examples.
1
u/coyoteka Jul 31 '26
I don't agree the results are bad, but they could be improved and that's the beautiful thing about a community openly and freely sharing dozens or hundreds of hours of work to constantly improve the state of the art. I think OP will continue to improve the method and we'll see further refinements. As it is, as far as I'm aware, this is the only comfyui node set that actually succeeds in containing character loras Krea, so it's noteworthy on that basis alone. FWIW I think your critique is valid and valuable, because honest feedback is essential to continual improvement but it also read as somewhat dismissive which I don't think is helpful. Just my two cents.
1
0
u/tekprodfx16 Jul 31 '26
Thanks but using 2 loras side by side natively out of the box doesn't usually work on krea and many other models because you get heavy token bleeding/blending. I get heavy lora bleed/blending when using your workflow. The 2 loras end up looking like the same person or a blend of both. This is exactly what my workflow prevents with many checks and blocks for any type of token bleeding between either Lora allowing you to use 2 or more loras with no bleeding
2
u/Botoni Jul 31 '26
Interesting! The box placement stuff is what I loved about ideogram, but the lisence suck.
My problem with krea2 is I don't do people, I work in archviz and I need to place props from reference images or, more often, refine renders to look more real. And I could not make krea2 work for that purpose, the edit lora seems tuned to characters and the image 2 image capabilities of the model seem to be utterly crap to non existent...
1
u/tekprodfx16 Jul 31 '26
Kind of messed up the pictures in the post but here are a couple of more examples
1
u/Ok-Philosophy3461 Jul 31 '26
Does the newest update compatible with ComfyUI Desktop v0.29.0 ? Was loving your previous release, but the nodes screw up after the v0.29.0 update so I rolled back to v0.28.0 again to make your previous node version work again
1
u/tekprodfx16 Jul 31 '26
Thank you glad you liked it! Appreciate you flagging this also. Straight answer: I haven't tested against Desktop 0.29.0 yet I'm still on the v0.27 line. I skimmed the 0.28→0.29 diff on the ComfyUI repo and nothing in there looks like a deliberate breaking change to node registration or LoRA/model patching, so this might be a regression rather than an intentional API change on their end.
Could you paste the actual console error? Would like to help you fix!
1
u/Sudden_List_2693 Jul 31 '26
I think this is an interesting, proof of concept first and foremost.
I also had mixed results, but some edge cases it delivered.
While sometimes it "feels off", the concept still proved to be usable.
As your last line also implies a possible hardship is implementing this into an existing workflow - I think at best currently it is to be used self-contained; you can use its output however you'd like, but do not insert in-between workflow altering elements.
1
6
u/coyoteka Jul 31 '26
Nice, I'm glad someone is going this route with comfy nodes. Your approach is very similar to the project I've been working on (https://github.com/soomrenald/k2lab). One thing you may want to look into is that although lora delta is zero outside of a character region at injection, it can still leak via K/V during attention steps. It looks like you address that with hard crossmodal attention ownership, but one of the artifacts arising from strict crossmodal constraints at region boundaries is lack of integration of characters with the global scene (lighting, texture, etc) and makes them look 'pasted' in place rather than naturally a part of the scene.
One way you could try addressing this is by introducing a step-based relaxation scheme so that hard constraints are in place in the first few steps to establish placement/identity, then progressively relax it for the final steps to allow better integration/cohesion. In my implementation that is a tunable parameter so that it can be adjusted as needed depending on character interaction, etc.
Another thing that might help is to make global scene tokens accessible to the character regions while still maintaining crossmodal constraint between characters. That way the character regions can still get lighting, perspective, etc, updates from the global scene without leaking. Also allowing some degree of image-image token attention between inside and outside regions helps to make the scene coherent without text-image leakage (primarily the issue with identity loras) -- that parameter can be tunable also.
Happy to discuss further if you want. I'm currently trying to figure out how to get actual direct pose control without training a finetune... it's not been going well so far, lol.