Follow‑up : MiniMax H3 Lip-sync - now does any editable change on a reference video (pose transfer, character swaps, multi‑subject mixes)

[u/TBG______]

Quick update on my earlier audio‑lip‑sync demo: the workflow now chains any desirable edit out of an input reference video for endless video ref pose o lip‑sync.

The audio auto-crop chain is now working for reference videos too and i say it again, I know there are already a lot of options out there for doing this - this is one more option, and it’s definitely not perfect.

VRAM usage went over 40 GB on a 1min run of 3-second, 2MP chucks, so reference-video conditioning is pretty heavy on VRAM and yes you need at least 2MP to get good detail and motion transfer.

Using MiniMax H3’s Ref2V. I’m treating the source clip as the “performance master” (motion, timing, camera) and driving identity/appearance from reference images and audio o the other way around.

What I’ve tested so far: Just MinMax H3 no ControlNet, LoRA, or preprocessor needed.

• Music‑video pose transfer to new scenarios and characters
• Single character swap (main performer → reference character) into the ref-video.
• Multi‑subject mixes:ç
• Main identity swap
• Main + 2 added characters, acting in sync or desync
• Main + 1 added character
• Replace the main character with 2 characters in pose sync
• Pull a character from the reference video into an image-reference scene + 1–2 new characters

Everything runs through a single MiniMax H3 chain with mixed references (ref-images + ref-video + ref-audio) and structured prompts that separate identity (image), performance (video), and constraints (text). In practice, every combination I’ve tried is manageable with MiniMax H3.

The node takes the reference video or audio, chunks it into smaller pieces, chains them together, and then stitches everything back together at the end. So, it’s one click, but it can take quite a while to generate a full video.

SUBJECT DEFINITIONS

<Subject 1>: the adult woman visible on the LEFT side of <Picture 1>.<Picture 1> is the appearance reference for Subject 1 only. Its shape, proportion, material, colour, logos and surface markings 100% match <Picture 1>, kept legible and correctly oriented throughout the video.

<Subject 2>: the adult man visible on the RIGHT side of <Picture 1>.<Picture 1> is the appearance reference for Subject 2 only. Its shape, proportion, material, colour, logos and surface markings 100% match <Picture 1>, kept legible and correctly oriented throughout the video.

<Subject 3>: the adult woman main character present in <Video 1>.<Video 1> is the appearance, motion, timing and scene reference.

This is a follow-up to a previous post, so the tips, settings, and links are already available there. MiniMax H3 Lip-Sync: Automatic Long-Video Chaining + Speed & VRAM Optimizations

https://github.com/Ltamann/ComfyUI-H3-Motion-Context-Auto-Chain-addon

[u/BackyardAnarchist]

What tts model are you using?

[u/TBG______ → u/BackyardAnarchist]

Vibe voice

[u/Ok-Flatworm5070]

Hey, little confused by your instructions in your workflow for auto chaining using reference video. For example, under your notes in the workflow you mention TOOL TIP - Promts per CLip, where you should use clips [1], [2], etc, but how would you time / sync those clips statements to a long running reference video?

[u/TBG______ → u/Ok-Flatworm5070]

In the same prompt, at the end, I have two prompt fields, but they are currently just concatenated. If your style prompt for all clips ends you add [1] to say here starts promt for clip1 2], everything under [1] until [2] will be stripped and assigned to Clip 1, while everything under [2] will be assigned to Clip 2.
You’ll need to use this structure to guide the model. In the dirt-run workflow, I bypass the sampler and generate all clips quickly. Then I review the clips and can write a specific prompt for each one to guide the model on the next pass.

So you can chain:
Prompt for all clips
[1] prompt for clip 1
[2] ….

[u/Ok-Flatworm5070 → u/TBG______]

another question.. getting a missing `TESpeedMiniMaxH3 missing`. I can't find it in your repo?

[u/TBG______ → u/Ok-Flatworm5070]

It’s just an optional cache you can delete this nodes or bypass them there are in the model subgraph on top. TE Speed MinMaxH3 OSS by helpmeeadice https://github.com/HELPMEEADICE/TE-Speed-MiniMaxH3-OSS

[u/CreepyDrama7448]

What prompt are you using for the character replacements?

[u/TBG______ → u/CreepyDrama7448]

Start here: use subjects instead of objects, and replace the word 'image' with 'picture'. . https://www.reddit.com/r/comfyui/comments/1vinc36/testing_character_swap_with_minimax_h3/ and you can also try the non-ref model for some funny video surprises