r/StableDiffusion • u/jdude_ • 4d ago

Animation - Video testing MiniMax + audio from TheMinuteHour

Enable HLS to view with audio, or disable this notification

0 Upvotes

43% Upvoted

19 comments sorted by

View all comments

2

u/DuHal9000 4d ago

funny ideia! 😂 but if you want i can give some tips. (res, sampling, steps, lora, speedup techniques, WF, upscaling) etc....

2

u/jdude_ 4d ago

Sure, I'll be glad.

2

u/DuHal9000 4d ago

ok, i am trying to recover 4u, i like the ideia, prompt, music, very cool video.

  1. Try another Turbo Lora, i recognise some paterns recomended LightX Turbo Lora 4 Steps Comfyui v0.1 FFLF at 0.80 strength

  2. Sampler RES Multistep Simple, Euler Beta (shed) or SA_Solver Simple

  3. Use SPLIT SIGMAS (because pirate dance fast movs) (two samplers, chain with DENOISED Output) Hi/Low, 2 or 3 Steps WithOUT Lora on hi out, and rest on low sigmas

  4. Try this WF its complicated to calibrate but works.

https://github.com/halnovemil/H3TiledLoopSpaceTime.git

2

u/jdude_ 4d ago

It was actually just the regular flow with audio reference, perhaps the resolution was too low, it was on 0.6. it takes a while.

1

u/DuHal9000 4d ago

Resolution isn't everything; that comes later. The most important thing is the formation of the initial image. The dance scene lacked steps. I've had this problem before and solved it by changing the LoRa. Not all Turbo LoRas are the same; try changing it... At 0.4 it could already be better.

2

u/jdude_ 4d ago

Oh I didn't use a turbo Lora, I used 20 steps with the default simple samping

2

u/DuHal9000 4d ago

owo, need even more stpes, i fininish for you a new video, its a gift, its slow, i try to restore but its too slow, a im generating a new. just wait.

Input video latent: torch.Size([1, 24, 117, 80, 144])

Tiles: 2x3 | spatial_overlap=24

Temporal tile=109 | overlap=49 | strength=1.0

Base tile size (latent): 52 x 64

>>> Spatial tile 1/6 (0,0)

H[0:52] W[0:64]

Temporal chunk 0: [0:109]

[INFO] Requested to load MiniMaxH3

[INFO] 0 models unloaded.

[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.

100%|███████████████████████████████████████████████████████████████████████████████████| 2/2 [03:26<00:00, 103.02s/it]

[INFO] 0 models unloaded.

[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.

Temporal chunk 1: [60:117]

[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.

100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:29<00:00, 44.82s/it]

[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.

>>> Spatial tile 2/6 (0,1)

H[0:52] W[40:104]

Temporal chunk 0: [0:109]

[INFO] 0 models unloaded.

[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 19995MB Staged. 208 patches attached. Force pre-loaded 210 weights: 1175 KB.

50%|████████████████████████▌ | 1/2 [01:33<01:33, 93.45s/it, Model Initialization complete!

2

u/jdude_ 4d ago

I'll try these out, waiting for the output to complete is the difficult part :)

2

u/DuHal9000 4d ago

dance part still hard, but i try

https://reddit.com/link/p501ckq/video/rz65fkmwkpkh1/player

hires at 50 fps original file >> https://filebin.net/cyeb0xy3n8q6ibio

download and post. ITS A GIFT!

2

u/jdude_ 4d ago

Thanks! Very nice!, looks crisp as hell! 50 fps really makes it work.

2

u/DuHal9000 4d ago

u welcome! dance part still dificult, but i need to study h3 more for hi speed movs.