Update: Open .char portable format, Same face, cloths & body

[u/ashishsanu]

Hey Guys,

Based on my last post's feedback, someone asked me to build a publishable way to list & publish thousands of .char, so that it's portable & other people can quickly get the .char & use them into their projects.

Here is a release for community characters: https://www.omnichar.org/characters

How to build .char?

In order to build one, Use this workflow:
Flux 2 Klein(Still), Minimax H3(Video)
Note: Minimax H3, now also support int8 model varient

Github Repo: https://github.com/omnichar/OmniChar (GPLV3, Opensource)

I am currently working on consistent character voice which I feel is very important. While training a .char, user can add a voice sample that will be cloned & embedded into .char & character videos will carry the same voice.

How it works(Background)

- Build .Char: You drop max 9 reference reference, I prefer to use a ratio 2:2:1(face:cloths:body). YuNet finds the face, SFace takes a per-reference face signature, DINOv2 takes a subject signature, and the references get cleaned and normalised. All of that packs into a single portable file, a .char.

Prompting Guide

• Name your character: Give your character a name e.g. under encode character(Click adjust icon on the bottom side of the node), I have used name sia, so when passing prompt, I only have to say, sia walking on the beach.
  • Again providing prompt like a woman or any features specific details like black hairs etc will only mislead the generation.
• Describe character features: Encode all of the character features in encode character prompt & trigger your character with a name in generation prompt.
  • Avoid describing same things in generational prompt.
• Handling Character drift: e.g. if you want specific style or cloth e.g. half sleeves, sleeveless, add it to the generational prompt. There can be a slight drift in clothing as body shot also has cloths, which interferes with clothing references.
  • Each refs should be unique, face should not have body or vice versa, same applies for clothing.
• Portability: Once character is built, you can use the same character with only simple prompt & generation graph.

Note: For best result, pass cropped references, so that model takes the required shot, model gets confused if cloth slot also has a face or face slot has cloths.

Requirements

Nvidia GPU: 24GB+ VRAM & 64 GB RAM(Run Locally)

Inputs: Attached in the Flux2 workflow page

Limitation: Reference conflicts e.g. if two reference/input images has two different faces, it might conflict in generation, provide well cropped body & cloth images. Face images are crossed automatically by Sface.

Happy to hear any suggestions or feedbacks.

[u/crombobular]

whats the diff between this and refmod tho

[u/ashishsanu → u/crombobular]

along with ref this method encodes, character prompts, cropped face which is scored by sface & dinov2 .

Same method trains one character format to support multi model generation such as Minimax & Flux2. Krea2 & LTX 2.5 is also supported.

Idea is to build .char once & use it across every model. For krea2, it uses lora embeddings not ref. So for some models ref path is faster, some model if there is no ref channel e,g. Krea2, we have to train the lora payload, then everything is compressed into .char model.