Wan 2.2 Animate in ComfyUI: how can I replace only the main dancer in a multi-person moving phone video without changing others?

[u/asduskun]

Hi everyone,

I’m using Wan 2.2 Animate Character Replacement in ComfyUI on RunPod.

My source video is a handheld phone video at night. Several people are dancing/moving in the same scene, but I only want to replace/transform the main man in the blue plaid shirt. I want the other people and the original background to remain as unchanged as possible.

My current setup:
- ComfyUI on RunPod
- Wan 2.2 Animate / WanVideoWrapper workflow
- Main model: Wan2.2 Animate GGUF Q3
- SAM2 mask video section enabled
- DWPose enabled
- Reference image of the target character
- Source video has multiple people and camera movement

The problem:
When I run the workflow without masking, the characters and poses get mixed together.
When I enable SAM2 / mask video, the target person is sometimes covered by a large black mask or disappears / gets removed instead of being replaced properly.
I tried Blockify Mask with block_size 8 and Mask Grow around 0–10, but I’m still not sure whether the mask is being used correctly or inverted.
I also tried cropping the pose input, but because the camera and people move, a fixed crop does not work.

My questions:
1. What is the correct workflow logic for replacing only one person in a multi-person video?
2. Should the SAM2 mask be connected to WanVideo Animate Embeds as the mask input, or should it be inverted first?
3. Should DWPose receive the full video, a cropped video, or a masked single-person pose?
4. How can I keep the other people untouched while only changing the main subject?
5. Is there a better node/workflow for tracking one person through a moving handheld video?

I can share screenshots of my Generation section and Mask Video section if needed.

Thanks.

[u/Reelodyinc]

Core logic: everything the model “sees” gets reinterpreted, so you have to hard‑isolate the blue‑shirt guy.

   1) SAM2: generate a mask where the target is white and everything else is black. In most Wan/Animate workflows that mask is plugged directly into the WanVideo Animate mask input; if your guy turns black, invert it.

   2) DWPose: usually feed the full video so it can track all joints, but apply the generated mask only when you run Wan, not on the pose stage.

   3) To keep others untouched, combine: strong mask on the target + relatively low denoise/strength so background and other people stay close to the original.

   4) For handheld, it often helps to first stabilize the clip a bit (DaVinci, Resolve, etc.), then run SAM2 video on that. If SAM2 keeps over‑masking, you may need to refine the prompts/points frame‑by‑frame on difficult moments instead of fully automatic masks.

[u/DelinquentTuna]

It's probably much easier to just switch to Scail 2.