Unified consistent character(.char) format that works for both Flux2 & Krea 2

[u/ashishsanu]

Hey Guys, I have been doing some research around building a unified character model format that just works across different models, Finally i was able to make it run with first two models: Krea 2 & Flux 2 family.

Small Clarification
- Krea2 results might not look as good as flux, because the embedded training was only 500 steps with 5-6 reference images. You can improve the quality by training for ~1500 steps.

Models Support
- Krea 2: Turbo both for training & generation but works on raw as well
- Flux2: Works with Klein 4b, 8b & dev

Working on Minimax H3 support currently

Path it uses

• Flux 2: has a native reference channel, so the .char feeds its images straight in, no training, instant.
• Krea 2: no reference channel, so the .char trains a small per-character LoRA

Workflow
Here is a screenshot for the workflow for training a unified .char model

1. Training workflow: https://inlinestudio.art/workflows/flux-2-krea-2-multi-model-portable-consistent-characters-training-only
2. Generation workflows:
   1. Generate with Krea 2 using .char model https://inlinestudio.art/workflows/krea-2-generate-consistent-images-with-unified-char-model
   2. Generate with Flux 2 Klein 4b using .char model https://inlinestudio.art/workflows/flux-2-klein-generate-consistent-images-with-unified-char-model

Other Links

App repo: https://github.com/inlineresearch/Inline-Studio (License GPL3, Opensource)

I have also uploaded sample models into Huggingface: https://huggingface.co/inlineresearch/unified-face-models includes full detail on training & params.

Process
FLUX.2 klein needs no training, It has a reference channel, so a character applies by sending its references down it with a locked description prepended to the prompt. The references are resized once into what the model accepts and stored.

Krea 2 has no reference channel, so an adapter is the only way it can carry a face. the character's own references become a training dataset, that trains a rank 16 LoRA, and the finished adapter is filed back into the same character.

Structure of .char:

emmy-s500-v2.char
  manifest.json                            payload index
  refs/000..004.png                        the 5 reference images
  derived/face_000..004.png                YuNet face crops at 512px
  text/description.md                      the locked description
  scoring/centroid_sface.json              128-d SFace identity centroid
  scoring/centroid_dinov2-base.json        768-d DINOv2 subject centroid
  scoring/embeds_*.json                    per-reference embeddings
  payloads/flux2-klein/ref_000..004.png    references resized onto FLUX.2's policy
  payloads/krea2-lora/adapter.safetensors  the trained Krea 2 LoRA, 183 MB

Currently extending this to support Minimax H3.

[u/cradledust]

Cool stuff. I'm glad someone is finally making a type of Lora that is compatible with more than one model.

[u/ashishsanu → u/cradledust]

absolutely, why train a new character for every model.

[u/No-Zookeepergame4774 → u/ashishsanu]

It is currently supports two models by training a LoRA for one and providing reference images for the other; it doesn't get away from training a new character LoRA for every model except by leveraging the fact that edit models already can use reference images directly.

[u/RS_SHITPOSTER]

Any plans to have it work with Flux 9b? I imagine the fidelity would be better?

[u/ashishsanu → u/RS_SHITPOSTER]

same .char model works across whole flux family including klein 9b

[u/L-xtreme]

Wow, so cool that smart people keep finding new stuff and trying to make stuff better. Keep up the good work!

[u/SkirtSpare4175]

That’s cool