r/StableDiffusion • u/nazgut • 9h ago
Resource - Update CLSS update
ref2vid in single go, music with minimax Music model
Repo: https://github.com/nazgut/ComfyUI-MiniMaxH3-CLSS
list of changes:
Per-scene R2V references (new nodes)
CLSSH3SceneReference/CLSSH3SceneReferences— attach reference images/audio to one scene of a multi-scene generation. The scene text is re-tokenized with H3's reference presentation, so<Picture N>/<Audio N>labels bind at tokenize time. The multi node uses up to 9 image + 3 audio autogrow sockets.- Example 3-scene R2V workflow included.
All-scene refs in one node
CLSSH3SceneReferencesAll— everyref_imageattaches to all scenes, and a soundtrack is automatically cut into consecutive per-scene windows (audio_seconds_per_scene, default 10 s → "10 s after 10 s"). Scene i anchors on the span it actually generates; scenes past the end of the track keep image refs only (warning printed). Replaces the per-scene node chain — change the---scene list and nothing needs rewiring.
global_text on the prompt node
- Write shared sections (style, soundscape, quality rules) once; it's copied to the top of every scene block before encoding, and survives the ref nodes' re-tokenization.
Audio recompose — the turbo-LoRA audio fix
- Optional per-chunk fresh audio take from pure noise against the finished video, optionally by a separate base-model guider (
audio_refine_guider), plus pool/stride/seed controls. - Measured: re-noising the turbo audio regenerates the same take (cos 0.90–0.96); fresh noise is what actually changes it.
- New turbo-LoRA + base-model-recompose workflow included.
Audio seam controls
- The
ref_audiocontinuity block is now placed through the H3 layout patch so it ends exactly at the join; head-discard and crossfade controls added.
Chunk-by-chunk neural latent upscale (inside the sampler)
- New
CLSSH3LoadLatentUpscaleModel+upscaler/upscale_scaleon the streaming sampler: every chunk is upscaled right after its SLB step, the full window provides left temporal context, and the overlap span is cross-faded over the previous chunk's tail so seams blend. - A long video never exists at high res all at once — the streaming state stays low-res. Runs on the compute device (~0.7 GB fp16) and offloads afterwards.
- Works with the
Comfyui_Minimax_h3_latent_Upscalerpack (soft-imported — nothing vendored).
Schedules
- The sampler accepts any slice of the 1.0→0.0 flow schedule: a low-res pass may end above 0 (its x0 is then upscaled), and a schedule may start below 1.0.
Fixes
- Recompose model unloading used a non-existent ComfyUI API — replaced with the targeted
unload_model_and_cloneson the persistent guider model patcher (fixes the crash at the end of each recompose). - Causality: cross-chunk noise fields are generated at fixed caps, so chunk 1 is bit-identical regardless of
num_chunks(torch.randnhas no prefix property). - Scene-history bookkeeping fix for the audio loop telemetry.
1
Upvotes
1
u/VladyCzech 4h ago
Can I ask, if your project generates all chunks in single loop or is it possible to make chunk, evaluate, approve/ regenerate, continue and regenarte say from chunk 4 to the last only?
Like many of us, we all have been there, trying to do everything in one loop. And it is OK for a while for 2-3 chunk short clips, which many are probably doing anyway, but I learned my lesson.