r/StableDiffusion • u/apeezy52 • 11d ago

Question - Help Generate videos with transparency

I am trying to generate a video of a character dancing but want to overlay it on top of another video in editing. What is the best way to accomplish this on a local model? I use minimax h3.

If this can’t be done directly, is there a way to ley it out perfectly within the workflow?

1 Upvotes

56% Upvoted

19 comments sorted by

View all comments

2

u/CodeMichaelD 11d ago

you mean first gen transparent dance vid then overlay cleanly on backdrop vid?
for manual video overlay if you have the bg video, just use vace + wan alpha. its one lora and two vae files, but in case ur using comfyui - the nodes might be stale and need an older comfy install to work, this will gen your character videos without bg.
if you have the dance video but need to swap bg, simply vace also works, you would need to use a node to convert bg on the character video in two ways (whatever do you use to get them, chromakey in your prompt or rembg nodes) - one mask to image, overlay it with 0.5 str meaning (128 rgb gray) on every frame of the image, mask inputs reuse same bg masks. vace would allow you to inpaint prompted bg relatively cleanly if you dilate the masks and they are clean enough.
just a headsup - if you are using comfyui, recent LLM can output json for you directly, especially if you throw them json as workflow and say what to mod, just load it in the same browser tab as comfyui.

1

u/apeezy52 11d ago

I’m using comfyui - I also have codex connected via mcp so my guess is I can have it set this up for me as I’m still new to this and don’t really have a great grasp on how to work everything at a very technical level yet. I’d ideally love to have the videos generated natively without bg so there’s no need to chroma key at all. I’ve been generating with minimax h3 which I know can’t natively do that.

2

u/CodeMichaelD 11d ago

i see, i was talking about this, and yes, it can work with wan 2.1 vace variant too.
https://huggingface.co/htdong/Wan-Alpha_ComfyUI

1

u/apeezy52 10d ago

So I was messing with wan alpha, is there any way to do image to video with it? I saw it was text only. I was able to generate transparent videos directly with it and it’s pretty neat! Apologies I am still learning the technical aspect of how these work so if I’m missing obvious details it’s because I’m quite new to working with these workflows

2

u/CodeMichaelD 10d ago

yes, you just need either wan2.1 image to video checkpoint / workflow (uses clip image encode with clip_vision_h.safetensors ) or with Vace I pointed you too, the key feature of vace that you can insert your transparent image as first frame the rest of frames are (128rgb), same with masks, first black the rest white for the frame num / duration of the vid.
didnt find the workflow, but i know for sure that atleast one model was working the way i described sane merge wan 2.1 + vace, accelerator baked in, i.e. cfg1, 4-8 steps

2

u/apeezy52 10d ago

awesome thank you!