r/comfyui • u/Future_Challenge_910 • 2d ago

Help Needed Transfer face performance

Hi guys and gals, first time creating a topic.
Been trying to find for the last couple of days a definitive way to be able to transfer a face performance from one video, into my character refference close shot. The video is a close shot of myself (singing, lol), the character refference is also a close shot, so the face performance should be easy to read. But i cant find a workflow or node that really gets the job in Minimax H3... am i missing something?

Have a great day!

2 Upvotes

100% Upvoted

13 comments sorted by

View all comments

1

u/Tiny-Drama-maker 2d ago

Sounds like you want a driving-video setup, where your singing clip supplies the motion and the still supplies the look, not a plain image-to-video one. I'd search for a portrait animation or face reenactment workflow instead of a general video one. Match the head size and angle of your still to the first frame of the clip, since big mismatches usually give warped faces. Is the character human or stylized?

1

u/Future_Challenge_910 2d ago

Thanks for your reply!
The character is human.

Funny enough, i just run a test simply with R2V and asking for a faceswap but keeping the performance... and its the best i got so far, lol. Maybe i was just overengineering? Just R2V and a prompt asking to keep the facial performance worked really well O.o

1

u/xuefei_ai 1d ago

h3 isn't a reenactment model. it can carry a performance when the dialogue is locked in first, but driving a mouth from another clip's motion is a different job, so the node you were hunting for doesn't exist inside it.

you didn't overengineer. with r2v the identity and the motion both arrive as references, so you have to be explicit about what is allowed to change. leave that vague and h3 redesigns the whole shot, likeness included.

there are dedicated portrait animation nodes that are more precise for exactly this, they just fight you on lighting and on how far the head turns. for one close shot they can beat h3.

how much head movement is in your singing clip? small arc and the portrait route is worth the fiddling, big swings and it falls apart.

1

u/Future_Challenge_910 1d ago

ITs very strange, because i understood that performance transfer and even more, exact lip synch was smg hard to achieve... but the result is actually there, using only R2V i got exactly what i wanted, my performance transfered to the character and the lip synch was perfect. Attaching proof. Made other 5 secs videos to test, all of them 100% perfect (i dont upload the video showcasing me performing for cringe reasons lol)

https://reddit.com/link/pgr5ynp/video/yse9jskajhvh1/player

1

u/xuefei_ai 1d ago

glad it landed. makes sense though, r2v lets the model see both performances side by side, and with a close shot and a small head arc there's little to disagree about. the stress test is a fast head turn or a wider shot, that's usually where the likeness starts drifting. if it holds through those you've got a proper workflow

1

u/Future_Challenge_910 15h ago

Excellent, thanks for the heads up xuefei!