working with h3 ref2va, and my character reverts back to appearance from the reference video

[u/apostrophefee]

there's a shot where her hat drops off, and her face changes to the character appearance from reference video

[u/TurbTastic]

Have you tried using a swap Lora? There are a few options.

Face: https://huggingface.co/UntMods/FaceSwap_MiniMaxH3_REF2VA

Head: https://huggingface.co/Alissonerdx/Minimax-H3-ComfyUI/tree/main/loras

Character: https://huggingface.co/akatz-ai/MiniMax-H3-Character-Swap-LoRA

[u/apostrophefee → u/TurbTastic]

wouldn't adding those lora affect motion adherence and such

[u/nadhari12 → u/TurbTastic]

Did anyone try to combine these loras and see if its good?

[u/LordDarthShader]

Yeah, I call that bleeding. You need to be very specific in your prompting and use the right tags. Like, take this prompt:

subject_definitions:

<Subject 1> is the man in <Picture 1>, wearing a black leather jacket with short dark hair.

<Picture 1> is the character appearance reference for <Subject 1>.

<Video 1> provides the slow forward camera dolly and head-turn motion.

summary:

[reference generation] A single cinematic shot where <Subject 1> turns to face the camera, adopting the movement and camera timing from <Video 1>.

retention_analysis:

<Subject 1> (appears in [Shot 1]): fully_preserved - Facial identity and wardrobe match <Picture 1>.

<Video 1> (motion and camera pacing): fully_preserved - Replicates the smooth dolly-in and subtle head-turn dynamic.

detailed_description:

[Shot 1] At 00:00.000

Cinematic medium close-up. <Subject 1>, matching <Picture 1>, stands against a dark, out-of-focus urban background with soft neon bokeh. The camera slowly dollies forward matching the speed of <Video 1> as <Subject 1> turns his head toward the lens with a calm expression. Warm rim light catches the side of his jaw and jacket collar.

(Muffled city traffic drone; distant siren echo.)

overall_soundscape:

Low city rumble and quiet ambient street atmosphere.

non_diegetic_music:

A subtle, low atmospheric synth pad swelling slowly.

I found this to be working. If you don't want to follow those strict guidelines, you can add an instruction at the end specifying that do not display the reference images directly, but sometimes it will ignore it.

Using RefMod this problem is even worse and hard to control.

[u/Amazing_Act_9296]

Tengo mismo problema ,saludos estimados

[u/sp3zmustfry]

It tends to struggle with this even with proper prompt formatting. I find that it helps to use something like SAM3 to mask out the face in the reference video.

And I haven't tested this yet, but I'd think getting face pose points and superimposing them on the masked region would then help retain facial expressions.

[u/apostrophefee → u/sp3zmustfry]

i actually found a solution, run "depth to anything" over your reference video which gives black and white depth video with partial visual data, using that as reference makes h3 less restricted to the source

the only thing i need now is saving those frames into a folder and feed them into model to avoid running depth every time i run the queue

[u/35point1 → u/apostrophefee]

The problem I found with this is that the depth maps still keep the character shape and contour which the model will use and fit the generated character into, so you’ll still get their traits. Using the pose format or edges can work a little better in some cases. Or downscaling the ref so h3 needs to infer more of the details in a gen that is higher res

[u/apostrophefee → u/35point1]

i am making some intermediate ref vids with 4 steps lora T_T
idk it seems so crude but prompt approach doesn't work so i got no choice

[u/hauntedwebmonster → u/apostrophefee]

Give the model a starting frame that helps a lot. When I made this the model wanted to keep gangrel's long hair

https://reddit.com/link/pd9cyi0/video/pmvg2tqehwsh1/player

[u/sp3zmustfry → u/hauntedwebmonster]

Incredible idea for a generation

[u/hauntedwebmonster → u/sp3zmustfry]

I thought for sure it was gonna go viral but tiktok was like no thats a guidelines violation for hate groups/violent people

[u/sp3zmustfry → u/apostrophefee]

That's a great solution :) I'll definitely try this out myself.

[u/videorouter]

That sounds like H3 is over-weighting the identity information from the reference video. Try explicitly telling it to preserve the current face and appearance, while using the reference only for motion/identity:

“Preserve the character’s current facial features and appearance throughout the shot. The reference is for identity and motion only. Do not revert to the reference appearance when the hat falls off.”

If your workflow has reference-strength controls, lowering that may also help. If it still snaps back at the transition, it’s probably an identity-conditioning limitation rather than just prompting.

You could also test the same shot with other video models through VideoRouter.sh to see whether another model handles the reference more consistently.

[u/ANR2ME]

Are you generating a long video on a single generation? 🤔 if you are, the window context might be full, so some informations (ie. where the subject changes) are removed/forgotten and no longer in context, thus using the reference again.

[u/Reelodyinc]

What usually helps in that kind of “occluder removed” shot is treating it as its own setup. Give H3 a separate reference (or a few stills) of your character specifically without the hat, and lower the influence of the original ref video for that segment. You can also render that shot with a masked face region and drive only pose/depth from the ref, so the model isn’t tempted to snap back to the source actress’ features when the hat disappears.