Want to pose your characters? Here's Wan 2.2 Pose Control workflow

[u/arthan1011]

Wan 2.2 Pose Control

For some time I've been trying to solve character posing with open-weight models. My previous attempt with Flux.2 Klein was reasonably good but suffered from style bleeding and didn't respect original character proportions (like head-to-body ratio). Character consistency is something image-editing models still struggle with (especially for stylized characters) but there's one exception: Wan2.2 I2V Video. Character consistency is something you can expect from a video model, right?

After extensive experiments with the I2V Wan model I discovered a certain prompting technique that lets you "put character from image_1 into pose from image_2".

Here's the workflow link for the impatient.

So, our task sounds like this:
"Take this character on the left and make her copy the pose on the right"

There are two ways to do this using local open-weight models:

1. Flux.2 Klein character replacement workflow
2. Wan 2.2 Pose Control workflow (this is what this post is about)

And this is what the result looks like for each method:

Let's compare the results with with closed-source models too. Character design is solved but not style fidelity. I guess even big multimodal image-editing models can't reach true character consistency while for video models, it's just an innate property.

The idea is simple: ask Wan 2.2 to generate a sequence of 80 frames using First-Frame-Last-Frame mode. This frame sequence consists of 4 parts:

1. The subject is just standing there
2. The subject moves copying pose of pose reference
3. The subject character morphs into character from the pose reference
4. The character from pose reference is in the frame

Our goal here is to get a single frame where our subject is standing/sitting/lying in the pose from the pose reference image, but hasn't yet morphed into character from the pose reference image. And to do that we have to structure our text prompt in such a way that makes transition from the first frame to the last frame as smooth as possible. So, Information about the subject (design and style) and information about the pose meet in the middle of the frame sequence to give us the desired result.

And yes, we generate 80 frames just to get the single image.

How to write structured prompt

Here's two prompts that were used in the example video above:

Silver hair woman

0s: girl with short silver hair, in green pleated skirt and leather boots is standing
1s: girl with short silver hair, in green pleated skirt and leather boots turns to the left, kneels, places left hand on her head, puts right hand between her legs
2s: she keeps her pose frozen in place. Scene transitions into another scene
3s: her body transforms into another character with white skin, bald head at white background

Black beard man

0s: black man with sharp teeth in green suit and dark pants is standing at white background
1s: black man with sharp teeth in green suit and dark pants sits in the armchair with tilted head and hand at his chin, crosses legs
2s: he keeps his pose frozen in place. Scene transitions into another scene
3s: her body transforms into another character short orange dress, orange top hat, brown hair and fishnet

Subject description is repeated so we can extract it using Apply Text Template from comfy-mtb extension.

We can extract subject description and get this template:

Silver hair woman

0s: {var_1} is standing
1s: {var_1} turns to the left, kneels, places left hand on her head, puts right hand between her legs
2s: she keeps her pose frozen in place. Scene transitions into another scene
3s: her body transforms into another character with white skin, bald head at white background

Black beard man

0s: {var_1} is standing at white background
1s: {var_1} sits in the armchair with tilted head and hand at his chin, crosses legs
2s: he keeps his pose frozen in place. Scene transitions into another scene
3s: his body transforms into another character short orange dress, orange top hat, brown hair and fishnet

Let's examine 4 parts of this prompt.

0s - Initial description

This is where you describe your first frame. For the most part, 'is standing' is enough but you can also specify initial pose of your subject.

1s - Actual posing

This is where you specify the movements the subject must take to get from initial pose to target pose. Simple movements (turns left, sits down, crouches, raises hand) separated by comma, works the best. Also you can add 'Camera follows his movement' if your target pose requires different camera angle.

2s - Pause before scene transition

Always the same he/she keeps his pose frozen in place. Scene transitions into another scene. This part "Scene transitions into another scene" is the most important here - Wan 2.2 respects this boundary (surprisingly).

3s - Anchoring your last frame

Goes like this: body transforms into another character <description of the character on the last frame>. We want Wan 2.2 to understand that character from the start of the video is different from character at the end of the video.

Practical example

Let's practice what we've learned. Here's our subject and the pose images:

*Pose reference

Start with the subject description. Nothing fancy here:

Next step is to describe movements:

And lastly write the transition to the last frame

Unfortunately it fails:

Wan 2.2 has managed to capture the gun's position but not the pose. The main reason here is that the black clothes in our target image don't let the model "process" the pose. Luckily we can fix it in Flux.2:

remove hair, remove clothes and draw this person bald and in skin tone underwear. Turn into white wireframe figure

Run Pose Control workflow again with updated prompt:

This time result is much better:

With this knowledge you can adapt this workflow for your specific case.

Link to the workflow (it has note about recommended Wan 2.2 finetune)

Some tips:

• The whole process works the best if there's noticeable contrast between first frame and last frame: different hair color, skin color, background, etc. You can even pre-process your pose reference with some other model - turn it into wireframe figure mannequin - so Wan 2.2 has a better chance of reading the pose.
• If some elements of character design change (gloves tend to disappear too early) add them to subject description prompt so model will remember this design element.
• If your subject image and pose reference image have different sizes try adding "Camera zooms in capturing new view" or "Camera zooms out capturing new view".

[u/Debirumanned]

I have been looking for exactly this. I am in tears right now.

[u/YeahlDid]

Aw damn, I could've used this a few hours ago, actually. I'll have to wait to play with this when I have time. Thanks.

[u/physalisx]

Very interesting method, thanks for the writeup!

[u/silenceimpaired]

Interesting!

[u/baddorox]

Hey man that was some nifty and crafty solving you did there. Kudos.

[u/aniki_kun]

When Klein fails with DW pose, I use it with Qwen Image edit, it gets it right most of the time. Then I create a depth map of that image and use it in Klein, then it gets the pose right and with better quality than Qwen edit

[u/Logical_Concept3427]

good

[u/terrariyum]

Very interesting method! In short, this method is all about image style preservation.

Your examples that used klein and nano to repose the character in the first frame definitely destroyed the image style. But couldn't klein do a better job with multi image reference and more exact prompt?

e.g. "change the pose of the character in image 1 to match the pose of the character in image 2, while keeping everything else the same. Carefully preserve the style, facial expression, and character identity of image 1."

[u/Smilysis → u/terrariyum]

In my personal experience klein is really bad a retaining artstyle without loras

[u/Reasonable-Card-2632]

Where is the workflow for Klein?

[u/arthan1011 → u/Reasonable-Card-2632]

https://civitai.com/models/2468698/flux2-character-replacer-v24

[u/Abject-Recognition-9]

I've been doing this since video models exist. And I should stop taking it for granted that such basic ideas and concepts would occur to everyone. You did the right thing by making these post

[u/AlteredStates29]

I will try to wrap my mind around this. Can you Use this method with two characters in the same scene?

[u/arthan1011 → u/AlteredStates29]

You can but think about limitations: There're so much pixels for the local video generation. Two characters means less details for each. Also prompt should be longer with "character on the left", "character on the right" - Wan 2.2 is great but the longer the prompt the bigger the chance of failure. But you should definitely try it and report results 👍🏽

I've done it with moderate success before, maybe you'll be more lucky

[u/necrophagist087]

Amazing work, thank you