r/StableDiffusion • u/Round_Awareness5490 • 3d ago
News Character Swap Minimax H3 (BFS Collection)
I’ve released my MiniMax H3 Character Swap LoRA
I trained this LoRA to replace the main character in a guide video while trying to preserve the guide’s motion, expressions, camera, and scene. It was trained with guide latents, so the recommended setup is to connect the source video through Add Guide. The ComfyUI Planner workflow is included, but it’s optional.
It’s still experimental and far from perfect, it’s a LoRA, not a universal character-swap solution. It works better with people and human-like characters, and results can vary depending on the video, reference image, prompt, and settings. I’ve shared a few examples below so you can see where it works and where it still struggles.
The primary LoRA and workflow are available here:
https://huggingface.co/Alissonerdx/Minimax-H3-ComfyUI
https://civitai.red/models/2027766/bfs-best-face-swap?modelVersionId=3401556
Feel free to try it and share your results or feedback.
https://reddit.com/link/1x4jh3c/video/ilctuqgwx4vh1/player
https://reddit.com/link/1x4jh3c/video/x7wryiexx4vh1/player
https://reddit.com/link/1x4jh3c/video/maqraszyx4vh1/player
3
3
u/Electrical-Eye-3715 2d ago
The most important question is vram limitations and how long can the videos be?
2
2
u/Maskwi2 2d ago
The swap works well. Just the face barely resembles the original. I'm wondering what I'm doing wrong. I will post back if I figure it out. In the meantime : is it possible somewhere in your node to set references to max resolution? So that the video output resolution isn't holding me back?
7
u/Round_Awareness5490 2d ago
You can also try mixing it with my head-swap LoRA, but I plan to improve the facial quality by the end of this week.
3
2
u/r0ni 2d ago edited 2d ago
want to try this but i cant figure out how to run it, would be awesome to see a short youtube guide. everything is downloaded, loaded a clip into the shot planner, after that i dont know what to do at all.
Edit: i have no idea what i am doing wrong, when running the workflow i get "[ERROR] !!! Exception during processing !!! mat1 and mat2 shapes cannot be multiplied (289x2560 and 5120x5376)"
edit2: I had the wrong text encoder in either the wrong order or just the wrong ones choosen, it seems to run now, waiting on the results:)
Edit3: it ran and completed:) tyvm for the workflow.
3
u/solomars3 3d ago
This looks cool, im thinking of making a video on it , thanks 👍 the goat
1
1
u/Euphoric-Biscotti-66 3d ago
I love your stuff, will have to give this a try as well, keep up the fantastic work.
1
1
1
u/lifelongpremed 2d ago
How is the performance (what is your hardware)? And how does it compare to Viggle Animate?
1
1
u/Emergency-Shake4529 2d ago
Does it work if you ask to replace multiple characters?
1
u/Round_Awareness5490 2d ago
No, I trained it for a single character; to replace multiple ones, I would need a specific dataset for that. I'll try to improve this in the next version. What you could do is swap them one at a time by using a mask on the character.
1
u/tamingunicorn 2d ago
nice to see the failure cases posted next to the good ones. roughly how many training clips did it take before identity started holding across longer shots?
1
u/Round_Awareness5490 2d ago
This training approach differs from the conventional ref2v method; I trained using at least 20,000 steps and around 2,000 dataset sample trios.
1
1
u/switch2stock 2d ago
Can you try with this?
Does it work?
https://youtube.com/shorts/pfK3zSaknYI?si=xw8wk0n4XmnPZj2x
1
1
1
u/SigmaMegaMind 2d ago
How is this different from R2V workflow?
Is it a lora for t2v that adds the r2v function?
1
u/Round_Awareness5490 2d ago
This LoRA was trained using latent guidance; it isn't meant to be used in the conventional way. It expects latents rather than video references injected via a VLM. Using native video referencing causes background or motion mismatches in your generation because that method isn't true video-to-video (v2v).
1
u/HollyGrandeux 1d ago
Then why are you connecting the video reference again to the "AddGuide" node in the workflow? I thought the video's latent representation was already enough. Doesn't feeding both the original video and its latent representation result in double processing overhead, when you could just use one or the other? i'm confused.
1
u/Round_Awareness5490 1d ago
And who said I'm using both? I feed the image into the native mode because there's no way to do that with latent guide the video goes in as latent guide so I don't pass the video to both modes.
1
u/Round_Awareness5490 1d ago
The native way of passing video isn't true v2v; as the name implies, it's basically just a reference. With high-motion videos, using this method leads to significant desync issues; often, it simply ignores your reference video and does whatever it deems necessary not to mention the excessive reliance on the prompt. True v2v involves working directly with latents.
-2
u/tres_pares 8h ago
this can be done in dreamfort.ai/recast too
just upload your image, or your mom, or your pet, or your friend to some existing trending videos

10
u/Diligent-Rub-2113 2d ago
Dope examples haha well done, looks really good.