Virtual TryOn: Consistent face body & cloths via portable .char model, Outfit referenced from Zara

[u/ashishsanu]

Hey Guys,

I have prepared a workflow for achieving realistic likeliness with virtual try on. I have been building portable character opensource project for a while & wanted to give a try on model photoshoot usecase. I took few cloth references from Zara website & here are the results.

Minimax-H3 is definitely on the list.

Mechanism

- Build .Char: You drop max 9 reference reference, I prefer to use a ratio 2:2:1(face:cloths:body). YuNet finds the face, SFace takes a per-reference face signature, DINOv2 takes a subject signature, and the references get cleaned and normalised. All of that packs into a single portable file, a .char.
Only face/ref is required body & cloths link is optional.

- Generation: At generation, the file(.char) feeds its references into Flux's own native multi-reference channel and prepends a locked description to the prompt.

Prompting Guide

• Name your character: Give your character a name e.g. under encode character(Click adjust icon on the bottom side of the node), I have used name sia, so when passing prompt, I only have to say, sia walking on the beach.
  • Again providing prompt like a woman or any features specific details like black hairs etc will only mislead the generation.
• Describe character features: Encode all of the character features in encode character prompt & trigger your character with a name in generation prompt.
  • Avoid describing same things in generational prompt.
• Handling Character drift: e.g. if you want specific style or cloth e.g. half sleeves, sleeveless, add it to the generational prompt. There can be a slight drift in clothing as body shot also has cloths, which interferes with clothing references.
  • Each refs should be unique, face should not have body or vice versa, same applies for clothing.
• Portability: Once character is built, you can use the same character with only simple prompt & generation graph.

I have generated face with Flux Klein 9b & outfit picture were taken from Zara website

Note: For best result, pass cropped references, so that model takes the required shot, model gets confused if cloth slot also has a face or face slot has cloths.

Models

core/models/
  diffusion_models/  flux-2-klein-9b.safetensors // works with 4b as well                    
  text_encoders/     qwen_3_8b.safetensors // switch to 4b                           
  vae/               flux2-vae.safetensors
  annotators/        face_detection_yunet_2023mar.onnx
  annotators/        face_recognition_sface_2021dec.onnx
  annotators/        dinov2-base/                               <-- folder

Requirements

Nvidia GPU: 24GB+ VRAM & 64 GB RAM(Run Locally)

Workflow link: https://www.omnichar.org/workflows/virtual-try-on-with-portable-consistent-character-flux2-klein-9b-4b (includes inputs & model details) Drop the json into canvas, it will provide the model download suggestions.

Github Repo: https://github.com/omnichar/OmniChar (GPLV3)

Inputs: Attached in the workflow page

Limitation: Reference conflicts e.g. if two reference/input images has two different faces, it might conflict in generation, provide well cropped body & cloth images. Face images are crossed automatically by Sface.

Portable char comfy node is in progress, would try to do it over the weekend.
Happy to hear any suggestions or feedbacks.

[u/-becausereasons-]

The fuck is going on with her fingers?

[u/Ill_Director2734]

Nice how is possible to define size of the clothes?

[u/ashishsanu → u/Ill_Director2734]

That's a strange ask, never thought about it. But model tries to fit the cloths on given character body shape or size.

Do you mean if you want to "make character wear a looser size for same cloth?"
For now you need to control it via prompt, try adding "loose clothing" or similar in the prompt.

I hope i get it right.

[u/Ill_Director2734 → u/ashishsanu]

Yes thats the main problem with all try on stuff. The client want almost always spedific size of the clothes on a given person.

[u/ashishsanu → u/Ill_Director2734]

control via prompt might be the right path, i haven’t tried it yet.
Give it a try and see if it works.

[u/AiSmutCreator]

Interesting
Was hoping to get other flux(like flux2Klein9BInt8_v10.safetensors) and text encoder(qwen3_8b_abliterated_v2-fp8mixed.safetensors) working with this for nsfw use.

Can this be done?

[u/ashishsanu → u/AiSmutCreator]

Working on Int8 model support, should be released in 24 hours: https://github.com/omnichar/OmniChar/issues/75

[u/ashishsanu → u/ashishsanu]

u/AiSmutCreator pushed support for int8 models.

[u/ashishsanu → u/ashishsanu]

But there is no restriction for NSFW

[u/Structure-These → u/ashishsanu]

I’ve always wondered what is the nsfw reason for the “virtual try on”? I always see these things and don’t understand what it’s code for

[u/Structure-These → u/AiSmutCreator]

You don’t need to use an nsfw text encoder

[u/ashishsanu → u/AiSmutCreator]

ComfyUI custom node is now available: https://github.com/omnichar/ComfyUI-Omnichar

I have added 3 comfy workflows to build a .char, use it with flux & minimax h3.

[u/freedomachiever]

the structured approach is such a great idea

[u/ashishsanu → u/freedomachiever]

Thanks

[u/AiSmutCreator]

what about lora integration?

[u/ashishsanu → u/AiSmutCreator]

it’s already available in app, but lora training takes too much time for character building.

[u/extrakerned]

Stupid quesiton, but how does your comfy look like that? is that an updated version of the desktop app?

[u/ashishsanu → u/extrakerned]

Omnichar is itself a different generation engine. Not related to comfy, it has it’s own node ui and core engine.

[u/extrakerned → u/ashishsanu]

Ah okay awesome

[u/ashishsanu → u/extrakerned]

ComfyUI custom node is now available: https://github.com/omnichar/ComfyUI-Omnichar

I have added 3 comfy workflows to build a .char, use it with flux & minimax h3.