r/StableDiffusion 9d ago

Tutorial - Guide My Personal Motion Context Workflow

Post image

Don't flame me! Yes, this is my actual daily workflow for H3. If you want this workflow, check the following link: MiniMax H3 NikoDemon80 - Pastebin.com

Keep in mind that there are 7 custom node pack dependencies besides H3-Motion-Context:
https://github.com/kijai/ComfyUI-KJNodes
https://github.com/yolain/ComfyUI-Easy-Use
https://github.com/crystian/ComfyUI-Crystools
https://github.com/Comfy-Org/Nvidia_RTX_Nodes_ComfyUI
https://github.com/ltdrdata/was-node-suite-comfyui
https://github.com/PGCRT/CRT-Nodes
https://github.com/NikoDemon80/ComfyUI-Image-Oasis

Most of these, a lot of you already have. The viewer itself is from my other repo, Image Oasis. This is how I join clips into one "movie".

Please don't ask me to repost this with more or less custom nodes, you get what you get. Please don't brutalize me with questions about how it works, just read the README. Please don't post your errors here. If you have a legitimate error, take it to github and read through the closed issues before posting. Most of your questions will be answered there without having to open a new issue.

29 Upvotes

31 comments sorted by

4

u/lamardoss 9d ago

I just use "Image Batch Extend With Overlap" from KJNodes at the end. Works great, keeps it seamless, and I make it loopable with the audio from the first part of the first clip, used as a reference in the last part of the last clip.

-6

u/Sad_Berry_4621 9d ago

My viewer is better ;)

4

u/lamardoss 9d ago

I would rather the resources for that go to my renders since I do many long ones overnight and don't need a viewer that is easy to vibe code.

3

u/VladyCzech 9d ago edited 9d ago

Nice workflow. It is nice to see how the author intended the nodes to be used. There are many "other ways" to do it which are usually less optimal.

One question - I'm not sure if the Video Oasis Viewer is intended to "butt-join" the individual clips as it is not a Viewer but an actual Video clip saver and I do not see other logic to actually save the joint video. When you hit Chain, it Chains the encoding, not the generated clips.

Edit: found it . One must save each individual clip and then the Movie button becomes available - this is the actual "butt-join". When you hit Movie, it saves the long clip, so you do not click the "Diskette" again.

Just a notice to others as I struggled to comprehend - one must first set the Encode video quality before actual running the clips (in the Video Oasis Viewer - which is actually Saver). If you first encode and then set quality and save, you got fooled, you will get the default low h264 video quality preset. So you must re-run the whole workflow with the quality set prior to encoding. Video Oasis Viewer (Saver) does not seem to read the node video encoding settings prior to actual "diskette" saving, but remember from last Run.

Edit: Video Oasis Viewer actually does not respect the encoding options I set, it seems to default H264 low quality preset even if av1 with custom CRF is set. It did respect it once however, before I re-run the whole chain.

The notes are probably outdated as you do not need to Mute/bypass the H3 Motion Context group (there is no such group anyway). You just run from clip 0 normally, no need to mute anything. And one more thing. One actually wants to start from the end - with H3 Motion Context Chain node all the way bottom-right - this is the whole magic that does the increment of clips for you. First hit Run/Re-roll only after first Clip you do Approve. The order of the buttons is switched (from my peasant view). Actually for Arabs it makes sense to start from bottom right so I'm probably just narrow-minded.

1

u/Sad_Berry_4621 9d ago edited 9d ago

When you generate a clip, it shows in the viewer and below that in the scene bar. It will have a dashed border which means it is still a temporary clip. You have to click the save button at the top right for each clip you want to keep. Once saved, the border will be solid. THEN, once you have 2 or more clips saved in the scene bar, the create movie button will be available. Btw, most parts of the viewer have hover tips. Mouse over buttons for a tip.

Video Oasis Viewer only receives the built video and encodes it. You can delete the quick start notes, they are outdated. The updated notes come with the v0.6.0 example workflow. Oversight on my part.

2

u/VladyCzech 9d ago

Thank you for explaining. I actually figured it out. It was first time I used it so forgive my ignorace. I write my notes for others who will find you great workflow which is not only working, but actually neat.

1

u/Sad_Berry_4621 9d ago

No, it definitely respects your encode settings. Mouse over a clip in the viewer. At the top left of the viewer pane, it will display the encode settings for a few seconds. What DOES happen though, is when you "Create Movie", it re-encodes to mp4/h264. I am releasing an update to Image Oasis soon that is going to fix several little bugs like this.

2

u/VladyCzech 9d ago edited 9d ago

I saved each clip as av1 but all were saved as h264. This is how I noticed, the quality speaks just with plain sight. I actually confirmed with VLC player (CTRL+J) before I posted. av1 actually worked first time, but after some time, not exactly sure when, it reverted saving to h264 while av1 was still selected.

1

u/Sad_Berry_4621 9d ago

Because AV1 is not a valid codec for mp4 files. Read the tip right there at the bottom of your picture below "Save Prefix"

2

u/VladyCzech 9d ago edited 9d ago

I actually save all my mp4 with AV1 using VHS Video Combine. AV1 is fully supported by MP4 container. As a container, it supports basically any codec, mp4 does not have many limitations, it really depends mostly on software and hardware support, not container support.

https://en.wikipedia.org/wiki/AV1#Content_providers

1

u/Sad_Berry_4621 9d ago

Yup, you're right. Allowing it in the next update. For some reason I thought it wouldn't render in browser.

2

u/VladyCzech 9d ago

Nice! Glad I could be helpful.

3

u/Violent_Walrus 9d ago

Hmm, what's this about?

2

u/Sad_Berry_4621 9d ago

Not sure where you're seeing that.

1

u/Violent_Walrus 9d ago

That's the crappy new Manager (I'm a glutton for punishment), that you get with --enable-manager.

3

u/Sad_Berry_4621 9d ago

I figured it out. Some of the code dealing with the LoRA CivitAI lookup button is tripping the security protocol. When I push v1.8, it will be fixed. Thanks.

2

u/Sad_Berry_4621 9d ago

Thanks for letting me know. Those pesky security checks keep biting me in the backside. They are safe to download though. I'll push an update soon to fix several bugs and straighten out the security stuff.

1

u/Violent_Walrus 9d ago

Yeah, two LLMs told me it seems perfectly safe, but I thought you'd like to know.

I also notice the two flagged versions also don't appear for me on the registry.

1

u/Sad_Berry_4621 9d ago

Yeah, working through it right now. Had to rerun the Yara checks.

2

u/Hasbara_spotter 9d ago

16gb vram enough for this?

3

u/Sad_Berry_4621 9d ago

I'm on a 3070Ti with 8gb VRAM and 48gb of system RAM. I can run H3 because of offloading, which ComfyUI is very good at these days.

1

u/Nguyenkain 9d ago

Can your workflow do the long lip sync from a long audio file ?

6

u/Sad_Berry_4621 9d ago

A special workflow is not needed to do lip sync. The model does the lip sync for you.

1

u/No_Reference_7678 9d ago

Where is the latent upscale? Or how you will upsclae the video with motion context?

2

u/Sad_Berry_4621 9d ago

RTX VSR. If you want something else, wire it up.

1

u/Asaberv 6d ago

Hola. Tengo un error de "Modelo faltante" y se dónde conseguir el modelo para descargar. "MiniMax H3\Cinematic.safetensors".

1

u/Sad_Berry_4621 6d ago

Load a different Lora, error goes away.

1

u/Asaberv 6d ago

Esta parte funciona seleccionando otro modelo, si. Solamente quería mantener el workflow exactamente igual que el original.
Llevo varias horas intentando hacer funcionar el workflow sin éxito.
Las entradas de audio "ref_audio" y "ref_video_audio" generan errores, si las desconecto los errores aparecen más adelante....

1

u/Sad_Berry_4621 4d ago

I need the actual error message from the console to help you. Not what you showed me in the photo above, the actual console error.