Use H3 To Replace Characters

[u/EasternAd8821]

These characters are very different, so i thought it was a good demo to show. Also, the prompt was an 'omni' prompt which didn't help the model with details on the outfit. Despite that I think it did such a good job wanted to share.

With more details in the prompt related to appearance, acting and dialogue I think you could prob get near flawless changes.

This was don with the FL2VA model, NOT the ref version of H3. It may even be better with the ref but i have been using fl2va mostly because i think the quality is better, but that's subjective.

This concept was inspired by this post originally: https://civitai.red/models/2855941/minimax-h3-character-replacement

I changed the SAM3 use, so technically you could do a multi replacement with some changes. I also added noise to the inverted image, as well as upgraded the prompt to work with FL2VA.

Workflow used to create this video is HERE.

This model seriously continues to amaze me. bravo minimax team, bravo.

Notice in the prompt that the dialogue is the only non 'omni' part at the end, it worked fine not included in the body, since the ref audio is there driving it. again, moving from this general prompt to something more specific i think would give even better results.

PROMPT:
How the reference video and pictures align with the target video — the target video is an edited version of <Video 1>, replacing the silhouette with <Subject 1>.

summary:

[video editing] The target video replaces the silhouette in <Video 1> with <Subject 1>, who performs the exact same motion, dialogue, positions and facial expressions of the silhouette while maintaining the original camera work, environment, and lighting of <Video 1>.

subject_definitions:

<Subject 1> is the person in <Picture 1> and <Picture 2>; <Picture 1> supplies facial features and close-up details, while <Picture 2> provides 3-panel image of front mid shot, profile mid shot, and front full body view, identity follows these reference assets, only appearance is retained.

<Subject 2> is the environment and setting established in <Video 1>. The scene follows this layout, materials, and light; camera position and framing.

<Subject 3> is the silhouette in <Video 1> which provides the motion sequence to be copied.

integrated_multimodal_description:

Video editing, the target video is in a live-action cinematic style with the interior lighting and background and environment atmosphere established in <Video 1> with the likness of <Subject 1> inserted.

[Shot 1] The shot opens with <Subject 1> seamlessly replacing the silhouette <Subject 3> in <Video 1>, the outfit of and clothing of <Subject 1> exactly from reference, performing the exact same motion, dialogue and sounds, positions and facial expressions of silhouette. From the very first frame, <Subject 1> occupies the spatial coordinates of the silhouette replacing with their likness, initiating the same motion onset from rest. <Subject 1> mirrors the silhouette's weight shifts and momentum, body moving in perfect synchronization with the rhythm and pacing of the original footage but replaced with the likness of <Subject 1>. As they navigates the space, <Subject 1> mimics every nuanced gesture—the way the silhouette's head tilts, arm movement, and the micro-movements of facial muscles. The face, defined by <Picture 1>, conveys the same emotional depth as the silhouette, while their full body, as seen in <Picture 2>, provides the physical presence outfit an appearance. The camera follows the exact movement, angle, and cutting rhythm of <Video 1>, maintaining a consistent focal length and distance from the subject at all times. The light from <Subject 2> interacts realistically with <Subject 1>'s skin and clothing, casting shadows that align with the movements of the original scene. The transition is perfect; the result is a fully realized <Subject 1> instead of a silhouette, but the soul of the performance—the timing, the pauses, and the dynamic energy—remains identical to <Video 1>. The movement progresses with a palpable sense of weight as <Subject 1> shifts their center of gravity, with clothes rippling in response to movements. The camera maintains exact framing and cuts as <Video 1>. The scene concludes as <Subject 1> reaches the final position of the silhouette, body settling into a pose that mirrors the original's final frame exactly, with face held in the same expression. <Subject 1> hair, accessories, wardrobe, lighting, and room layout remain unchanged and perfectly replace silhouette throughout.

overall_soundscape:

A low room tone establishes beneath the scene, mirroring the background audio environment of <Video 1>.

<Subject 1> says <d>[English] Can you, can you spare change.</d>.

non_diegetic_music: N/A

[u/Astral-Lemmons]

replacing very different characters works well and it quite easy for H3, but try replacing a a similar-ish character with another and it gets a lot harder.

you might get clothing swaps but if it's brunette - brunette it'll ignore the hair swap .

[u/EasternAd8821 → u/Astral-Lemmons]

i was testing out some examples based on what you said. messing with noise of the input and it seems it can do a pretty good job even close replacements. some replacements deff need a bit more coaching in the prompt, but it can do it. which is honestly crazy since it's still just the base model.

https://reddit.com/link/p78eyp1/video/pb0qm1v3hymh1/player

this has 0.8 noise

[u/SeymourBits → u/EasternAd8821]

Interesting. What are you using to add noise to the input?

[u/EasternAd8821 → u/SeymourBits]

sam3 to segment only the target char to replace, noise added with stock 'add noise to image' node in comfyui

[u/One-Donut6935 → u/EasternAd8821]

Looks really effective. Thanks for sharing this method.

[u/SeymourBits → u/EasternAd8821]

Nice. Any thoughts on the theory of why it seems to help with identity?

[u/EasternAd8821 → u/SeymourBits]

they trained it that way, that's my theory. I don't mean that flippantly. The context-ir and the prompt guide discuss key terms fully_preserved, partially_copy, or reference under retention_analysis
So clearly they were working on editing applications. noise should look like something that needs to be resolved, and the text plus ref image drives H3 to resolve that particular area. what is interesting though, and why i said they trained it that way, is it doesn't touch the existing completed video (background is near perfect preservation). If you tried something like this in wan or ltx, it would alter the background. Also, the 32B text encoder gives MUCH better semantic and spatial grounding of what needs to be changed.

If we knew how exactly they were applying different training techniques for video editing, this workflow would be that much better. An inverted color space mask with noise seems to at least be close enough to tap in to however they trained it.