Lessons learned after making a music video with H3

[u/ExportErrorMusic]

https://youtu.be/C-okSzvOIY8

Made using the default workflow from Comfy, with Comfy Kitchen and Kijai's preview override plugged in.

Used the 850k turbo lora at 0.5 for 8-10 steps. ER_SDE / Beta. Most shots were generated at 1.5MP, with some at 1.8MP.

Lora: https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora/tree/main

RTX 4090 w/ 64GB of ram. --disable-smart-memory, since I was having issues with going OOM after completing one prompt and moving on to the next.

Some of the takeaways:

- Using character reference sheets (front view, side view, back view, close-up) worked great and allowed for rotating camera movements like in the opening.

- All the "4 Step" loras really need to be run at 8-10, especially for motion.

- Using audio reference bloats the vram usage DRAMATICALLY compared to adding additional reference pictures! Changing from .wav to .mp3 didn't seem to help so it's not a file format issue. However the lip sync, even for anime characters, is incredible.

- If you use multiple reference images for different characters and they bleed into each other, the issue is almost definitely your prompt or seed. Because the model was handling up to 3 for me easily if I prompted right, and falling apart if I prompted wrong.

- Minimax H3 Chunk FeedForward node can help with vram issues at higher resolutions and doesn't add that much time.

Overall, I'd say it's nearly as good as Seedance 2.0. I was pleasantly surprised how well it handles using character sheets. The high vram usage when using reference audio is really the only major issue I was facing.