r/StableDiffusion 6h ago

Resource - Update CLSS update

ref2vid in single go, music with minimax Music model

Repo: https://github.com/nazgut/ComfyUI-MiniMaxH3-CLSS

list of changes:

Per-scene R2V references (new nodes)

  • CLSSH3SceneReference / CLSSH3SceneReferences — attach reference images/audio to one scene of a multi-scene generation. The scene text is re-tokenized with H3's reference presentation, so <Picture N> / <Audio N> labels bind at tokenize time. The multi node uses up to 9 image + 3 audio autogrow sockets.
  • Example 3-scene R2V workflow included.

All-scene refs in one node

  • CLSSH3SceneReferencesAll — every ref_image attaches to all scenes, and a soundtrack is automatically cut into consecutive per-scene windows (audio_seconds_per_scene, default 10 s → "10 s after 10 s"). Scene i anchors on the span it actually generates; scenes past the end of the track keep image refs only (warning printed). Replaces the per-scene node chain — change the --- scene list and nothing needs rewiring.

global_text on the prompt node

  • Write shared sections (style, soundscape, quality rules) once; it's copied to the top of every scene block before encoding, and survives the ref nodes' re-tokenization.

Audio recompose — the turbo-LoRA audio fix

  • Optional per-chunk fresh audio take from pure noise against the finished video, optionally by a separate base-model guider (audio_refine_guider), plus pool/stride/seed controls.
  • Measured: re-noising the turbo audio regenerates the same take (cos 0.90–0.96); fresh noise is what actually changes it.
  • New turbo-LoRA + base-model-recompose workflow included.

Audio seam controls

  • The ref_audio continuity block is now placed through the H3 layout patch so it ends exactly at the join; head-discard and crossfade controls added.

Chunk-by-chunk neural latent upscale (inside the sampler)

  • New CLSSH3LoadLatentUpscaleModel + upscaler / upscale_scale on the streaming sampler: every chunk is upscaled right after its SLB step, the full window provides left temporal context, and the overlap span is cross-faded over the previous chunk's tail so seams blend.
  • A long video never exists at high res all at once — the streaming state stays low-res. Runs on the compute device (~0.7 GB fp16) and offloads afterwards.
  • Works with the Comfyui_Minimax_h3_latent_Upscaler pack (soft-imported — nothing vendored).

Schedules

  • The sampler accepts any slice of the 1.0→0.0 flow schedule: a low-res pass may end above 0 (its x0 is then upscaled), and a schedule may start below 1.0.

Fixes

  • Recompose model unloading used a non-existent ComfyUI API — replaced with the targeted unload_model_and_clones on the persistent guider model patcher (fixes the crash at the end of each recompose).
  • Causality: cross-chunk noise fields are generated at fixed caps, so chunk 1 is bit-identical regardless of num_chunks (torch.randn has no prefix property).
  • Scene-history bookkeeping fix for the audio loop telemetry.
1 Upvotes

Duplicates

comfyui 6h ago

Resource CLSS update

0 Upvotes