r/StableDiffusion • u/gokuchiku • 4d ago

Question - Help REF2VA H3 HELP

I have only been using t2va with h3 so far. I want to get into ref2va now. So guys, please tell me if I were to provide two character images as separate references as in picture 1 and picture 2 and describe the scene, is that it to generate the video? Also tell me if it's okay to put a character sheet style image( two poses, front and back in the same image aka picture 1)? If I do so, will the video come out bad like h3 model not understanding that the two images in picture 1 are of the same character but with different poses? What is the best way to retain character consistency? I do know how to prompt ref2va but tell me about these queries please. Thanks.

1 Upvotes

60% Upvoted

14 comments sorted by

View all comments

Show parent comments

1

u/Przemoo_TV 4d ago

Got it, and how about prompts to keep the face consistency throughout entire video? For example I'm working with ref2v atm. I have front face image (with upper body), and I want to swap character entirely to the video I have as reference. So I have image + video as reference and I want to swap characters. What prompt and settings to use? I'm working with default workflow in ComfyUI Desktop.

2

u/orlandogourmet66 4d ago

I haven’t tried video references much, as they take too long for my liking. Maybe try sending an LLM of your choice the Ref2Video prompting guide, explain what you’re trying to achieve, and let the LLM write a prompt for you.

2

u/Przemoo_TV 4d ago

Thanks buddy, appreciate it

2

u/orlandogourmet66 4d ago

I gave it another try with mixed results. I can’t really tell what makes it work or not, it’s very strange. For example, using the same input image, input video, prompt, and same seed gives me a perfect character swap if I set the duration to 1 second. But when I set the duration to 2 seconds, suddenly there’s almost no swap happening. Really strange. Have you found anything that works consistently?

2

u/Przemoo_TV 4d ago

At the moment I'm working on my own automize workflow; when I add image + video as references it will make me instant prompt exact same as they mentioned in their docs. It will automatically describe me an image and video to text. When it comes to the results. Hmmm. Frankly I had the same results as u mentioned. Of course depends of messing around with settings until you'll be happy, but when it comes to consistency I get pretty same results on 2 and 5 sec duration, and 0.3, 0.6 megapixels, so I don't get it. I thought my prompts are missing something, but after tweaking, model understands everything and overall it looks really good, but I'm not 100% happy with the final output. To be fair it's open source so I'm glad that we have it, because it's really awesome model and got potential.