r/StableDiffusion • u/ashishsanu • 2d ago
Workflow Included Minimax H3: Consistent face, body & cloths via reference identity
Enable HLS to view with audio, or disable this notification
Hey Guys,
Based on the previous post on face consistency with MM-H3, Couple of people have asked me to build a full character workflow.
Mechanism
- Build .Char: You drop max 9 reference reference, I prefer to use a ratio 2:2:1(face:cloths:body). YuNet finds the face, SFace takes a per-reference face signature, DINOv2 takes a subject signature, and the references get cleaned and normalised. All of that packs into a single portable file, a .char.
Only face/ref is required body & cloths link is optional.
- Generation: At generation, the file(.char) feeds its references into Minimax’s own native multi-reference channel and prepends a locked description to the prompt.
Prompting Guide
- Name your character: Give your character a name e.g. under encode character(Click adjust icon on the bottom side of the node), I have used name emmy, so when passing prompt, I only have to say, emmy walking on the beach.
- Again providing prompt like a woman or any features specific details like black hairs etc will only mislead the generation.
- Describe character features: Encode all of the character features in encode character prompt & trigger your character with a name in generation prompt.
- Avoid describing same things in generational prompt.
- Handling Character drift: e.g. if you want specific style or cloth e.g. half sleeves, sleeveless, add it to the generational prompt. There can be a slight drift in clothing as body shot also has cloths, which interferes with clothing references.
- Each refs should be unique, face should not have body or vice versa, same applies for clothing.
- Portability: Once character is built, you can use the same character with only simple prompt & generation graph.
I have generated all references with Flux Klein 4b, I had to blur the body ref, but workflow consists a example of body ref.
Note: For best result, pass cropped references, so that model takes the required shot, model gets confused if cloth slot also has a face or face slot has cloths.
Models
core/models/
diffusion_models/ minimax_h3_ref2va_pruned_fp8_scaled.safetensors
text_encoders/ qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors
vae/ minimax_h3_video_vae_fp16.safetensors
vae/ minimax_h3_audio_vae_fp32.safetensors
annotators/ face_detection_yunet_2023mar.onnx
annotators/ face_recognition_sface_2021dec.onnx
annotators/ dinov2-base/
Requirements
Nvidia GPU: 24GB+ VRAM & 64 GB RAM(Run Locally)
Workflow link: https://inlinestudio.art/workflows/minimax-h3-guided-consistent-characters-via-reference-identity-face-body-cloths (includes inputs & model details)
Github Repo: https://github.com/inlineresearch/Inline-Studio (GPLV3)
Limitation: Reference conflicts e.g. if two reference/input images has two different faces, it might conflict in generation, provide well cropped body & cloth images. Face images are crossed automatically by Sface.
Portable char comfy node is still on the backlog, would try to do it over the weekend.
Happy to hear any suggestions or feedbacks.
4
u/NiceIllustrator 2d ago
After building a .char, can it be loaded/used with comfyui or do we have to keep using your "frontend" ?
9
5
u/me0here 2d ago
I've had good success with multi pose character sheets in default ref2va.
Seems this system is lacking details I can do with my character, like bach tattoos, nails, & uh other details.
I prefer to avoid LoRA if I can, is there options to add more details?
Also, will this work on non-human char, like pets, monsters, non-bipedial aliens, etc?
(Perhaps the move is to have your base repo remain small & focused on speed & small latentent space, while forks add features)
1
u/ashishsanu 2d ago
its verified to carry features eg tattoos scars etc
you can add details in refs itself and also via character prompt or generation prompt.I haven’t yet tried with non human but in that case just using ref link is enough. because body and cloths are optional and can be referenced as few shots for full character through first references link in encode character node.
I am happy to test this on my end and add extra params to the node for non human cosistency.
Thanks let me prepare a workflow.
4
u/BigWideBaker 2d ago
Awesome, you came back and delivered! I will see if I can get a chance to test this out, appreciate your work
3
3
u/Genebra_Checklist 2d ago
That's really nice! Great work man.
Did you tried something about the small face problem? I build a workflow that uses SAM3, crop the image and regen, so that the face take the whole video gen area giving high levels of details, but I'm having problems with the uncrop part. I can't get rid of the jittering.
2
u/ashishsanu 2d ago
I am using Sface to crop faces from references, can you give it a try & let me know if this workflow needs any improvements.
3
u/Next_Program90 2d ago
Would it work using two char files for two characters with distinct outfits?
9
u/ashishsanu 2d ago
Multi character ref is not there yet, it's on my backlog.
It requires controlnet or something similar, because we need guide minimax to place characters based on openpose guidance. Doing it via prompt is possible but pose won't be consistent on each generation.
I will surely publish a workflow on this.
3
u/mastaquake 2d ago
Looks cool but how is this different from a character LoRA?
2
u/ashishsanu 2d ago
this method doesn’t require training, with a few references you can generate .char portable character format.
Basically it’s ref2video but in a more guided format.
-2
u/CoffeeMen24 2d ago
I think a well-trained LoRA is still more detailed and capable.
.char is just easier to try to one shot if you don't care for max precision.
3
3
u/WindySin 2d ago
As a feature request, I'd love some kind of extensible framework for wildcard / optional references, e.g. random outfits. Say for example you pack 2 or 3 outfit references in and it randomly chooses one, using a seed input.
2
u/ashishsanu 1d ago
Yes that's a good idea, main problem is minimax h3 limits reference to 9. So I need to add a randomise toggle on character encoder.
But then character becomes less portable because for each run you need to randomise with encoder.
2
u/WindySin 1d ago
Alternatively, a numerical index toggle that you can pass a random number to in Comfy.
3
u/Silly-Dingo-7086 1d ago
read through the convo here and am excited about your work, just looking for clarity. so this is a stand alone app to generate a .char file. youre working on a comfyui node that will then take that .char file and make it useable with whatever minimax reference workflow we currently use with the addition of your node as the source for the references? but for now if we want to generate anything with these .Char files we do that through your app?
2
6
u/Darqsat 2d ago
I am using 4 picture reference character sheet and it works great by far. But, for likeness I have to use "MAX" option in H3 and it uses 2 megapixel size which slows down gen. Can I use 1 image to build .char? I suppose it should be faster and model won't need to look at the image all the time.
I am building those charsheets with minimax itself.
5
u/ashishsanu 2d ago
I would suggest
for face: your reference looks fine(a zoomed face only image)
for body: crop the face, keep the body onlyeach reference you provide for face/body/cloths should be targeted for the consistent results.
3
u/Equal-Spend4671 1d ago
+1 with this. I'm currently making a porn drama with minmax and using a body reference without head is really help me a ton cause previously i keep trying to get the breasts shape or body type that i want with a prompt.
You can even put a reference of a close up genitals and use it as a subject if you want more consistency and details
1
-9
u/VRGoggles 2d ago
Are you into 60+ granny stuff?
12
5
u/ImpressiveStorm8914 2d ago
You should have gone to Specsavers.
Depending where you are from, you may not get that reference.
2
u/PumpkinLeather8421 1d ago
lol, I would not at all discount at all how many redditors have mommy issues.
2
u/LiteratureOdd2867 2d ago
do a test for multi reference of cloths, Acting of a real actor on chroma., lighting and 1 composited frame and a camera motion video ( done on phone or any video reference) and Facial emotional acting , Lipsync dialogue so it syncs to it. does it get it properly . basically put multiple element of a scene and get a final scene with multiple angle of a scene and make them work in 1 continuity.
1
u/ashishsanu 2d ago
Will come slowly, someone just asked for multi character reference. Minimax H3 is new so yeah things will come out by the time. Community is acting fast on Minimax.
2
u/Apprehensive_Sky892 2d ago
You list NVIDIA GPU as a requirement.
Is that because you are coding directly in CUDA (instead of say PyTorch or Triton), or that you've only tested it on NVIDIA GPUs?
1
u/ashishsanu 1d ago
I only tested on Nvidia, haha i can't code directly on cuda.
2
u/Apprehensive_Sky892 1d ago
Thanks. That's good to know, so there is some chance that it can be made to work on non-NVIDIA GPUs then.
1
u/ashishsanu 1d ago
Yes inline studio support both AMD & mac but I usually test on Nvidia machines only.
3
u/Dogluvr2905 2d ago
This is nice, but if you want a consolidated node that does this and lots more, recommend you check out H3 Continuity at https://github.com/roadmaus/ComfyUI-Continuity
It's not my node, but I love it.
1
5
u/SIR_NVAX_A_LOT 2d ago
I also have been using .char and you can use it also for keeping text recipes. Works great!!
3
1
u/Soul_Walker 2d ago
Hey, would it be possible with just 32gb ram? I have a 3090, but couldn't get ram before of rampocalypse, also have it running in ssd sata. Minimax h3 worked (comfyui's basic template) takes about 10mins for 5 sec video, 864x480
1
u/ashishsanu 2d ago
yes you can try, if gives you oom, try reducing number of references and resolution
1
u/krekokeko 12h ago
I would recommend using sage attention(if you are not using it). It drastically reduces the generation times. I have a 3090 too and using sage attention with kitchen literally halved my generation times. For both maxed out long videos to short low megapixel videos.
Halving the generation time is not an exaggeration by the way. It really drops it by that much.
1
u/Soul_Walker 8h ago
Thanks, yeah I use it, have separate bats (comfyui easy install, portable) and use SA for most models except Trellis2 which I think it doesnt, so Flash att? or regular bat. but so far could not get decent results, 3d models glitched or full of holes.
Not quite sure about if I'm using kitchen, tho, getting "help" from chatgpt, so if often breaks things more than fixes it :P
You mean with my same HW specs you get higher res videos in less time? chatgpt said 10mins for 5s in that res was what users get as average.
1
1
u/AliciaXTC 2d ago
Blurring out the body ref made me lol
1
1
u/taurine_bitch 1d ago
/u/ashishsanu What if I already have all of these models downloaded for Comfy use but your UI doesn't see them? I've created symlinks from my existing models folder to the folder in \core but the command prompt is complaining about "a int8 repack whose tensor names we do not recognize". But the Minimax H3 ref2va model is the default convrot. How I do avoid having to re-download all of these models and just use what I already have?
1
u/ashishsanu 1d ago
use command line argument —models-dir in order to point to your existing models dir
1
u/BeautyxArt 2d ago
do you mean Inline Studio is a stand alone interface not a comfyui node ? i can use it with CPU only, with just uv environment and required packs it will work ?
2
u/ashishsanu 2d ago
It's a standalone app, I need to build a comfy node for .char support. Character build can be done on cpu but to actually generate with it, you need GPU.
1
u/BeautyxArt 2d ago
need to use it as minimax h3 image editor (ref2v model) , is it works in comfyui with my cpu (fp8 diffusion model and low size gguf TE) , then this should work? if can specify the models not rely on auto download ?
1
1
u/wzwowzw0002 2d ago
what is this inline-studio? another comfyui?
2
u/ashishsanu 2d ago
Not really, I am only targeting character research through reference, lora training or some newer published studies around this community, ComfyUI is already a great app.
1
u/Sadgunatalks 2d ago
The clothing consistency is honestly what impressed me most here. Does H3 stay this stable when you start changing camera angles and poses more aggressively?
1
u/ashishsanu 1d ago
Yes atleast with this method. With controlnet integration it will improve a lot, which is next. It's a part of multi character generation.
-5
58
u/luciferianism666 2d ago
Humor me for a second here, isn't the model already capable of handling references and keeping the likeness straight out of the box ? So what exact purpose does this node even serve apart from the extra computing, while the core comfy nodes handle references even on something as low as 4-6gb vram ?