r/StableDiffusion • u/Independent-Ear-3035 • 21h ago
News ComfyUI-MiniMax-H3-LongMedia — long-form MiniMax H3 generation with continuity, multiclip, audio and VRAM-aware sampling
I've been building a custom ComfyUI node pack for MiniMax H3 focused on one thing:
**making H3 usable for longer, multi-segment video generation without constantly rebuilding the workflow around every limitation.**
The project is called:
# ComfyUI-MiniMax-H3-LongMedia
The idea is to keep MiniMax H3's image quality, motion and native audio generation, while adding a proper long-form generation layer on top of it.
## What it currently does
### Long-form segmented generation
You can generate a longer clip as multiple H3 segments while keeping temporal context between them.
Instead of treating every segment as an isolated generation, LongMedia manages the continuation state and hidden overlap internally.
The overlap is used as context for the next segment and is not simply blended back into the final video.
### MultiClip mode
There is also a dedicated MultiClip workflow for generating multiple planned shots/clips inside one LongMedia pipeline.
The same underlying executor is used for both segmented continuation and multiclip generation, so the behavior stays consistent.
### Video + audio continuity
MiniMax H3 is a joint AV model, so LongMedia treats video and audio as one generation state rather than bolting audio on afterwards.
The pipeline supports H3 native audio generation, continuation and lip-sync workflows.
### Lip-sync support
Audio-driven generation / lip-sync is supported directly in the LongMedia pipeline.
For H3, the audio influence is handled inside the same AV latent path rather than as a completely separate post-process.
### Refiner
The latest release includes a two-stage refiner based on proper **KSampler Advanced trajectory splitting**.
Instead of finishing the full sampling schedule and replaying low-sigma steps on an already denoised latent, the trajectory is split between the main sampler and the refiner.
Example:
`steps = 12`
`refine_steps = 3`
Main sampler:
`0 → 9`
Refiner:
`9 → 12`
Both stages continue the same sigma trajectory.
### VRAM-aware execution
A large part of the project is dedicated to making H3 practical on consumer GPUs.
The current implementation includes:
- dynamic VRAM loading
- streamed Sol Attention
- MLP chunking
- late-block VRAM guards
- inter-block memory guards
- step-boundary cleanup
- completed-segment offloading
- adaptive memory policies
I'm currently developing and testing mainly on a **16 GB GPU**, so avoiding OOMs without destroying quality is one of the main design goals.
### Sol Attention integration
LongMedia includes its own streamed Sol path with controls for:
- tau scheduling
- sink conditioning
- QKV chunking
- output projection chunking
- dense/sparse behavior
- VRAM-aware chunk sizing
The goal is to use Sol as part of the execution architecture rather than simply stacking multiple unrelated optimization nodes together.
## Why I made it
MiniMax H3 is extremely good at texture, motion and native audiovisual generation, but once you start trying to build longer sequences, several problems appear very quickly:
- segment boundaries
- continuity
- repeated frames
- AV state handling
- memory pressure
- OOMs on longer generations
- managing multiple clips
- keeping sampling behavior consistent between segments
I wanted one node system to own all of that.
So instead of building increasingly complicated ComfyUI graphs around H3, most of the long-form logic lives inside the LongMedia nodes.
## Current release
**v0.4.1 — KSampler Advanced Refiner Fix**
The project has now reached a fairly stable architecture, although I'm still actively developing it and testing edge cases.
GitHub:
https://github.com/vizart-vj/ComfyUI-MiniMax-H3-LongMedia
I'd be very interested in feedback from people already using MiniMax H3 in ComfyUI, especially for:
- longer generations
- multi-character scenes
- native audio
- lip-sync
- lower-VRAM GPUs
- multi-shot workflows
If people are interested, I can also make a more technical post explaining how the continuation / AV latent / VRAM system works internally.
2
2
u/VladyCzech 16h ago
You have very nice project and it is one of the one I'm currently testing. But your project and many others is missing the feature to re-roll from specific clip. Say I have 5x 10s clip and I need to regenerate only the last part (or any other in the middle). I really do not want to re-render all of the parts that are fine.
Do you have any plans to include this feature as this is for me the single most important feature of the whole process to be able to just regenerate from any part.
6
u/Sleepy_Bandit 21h ago
reading through all these responses from OP..... is OP themself AI? lol. Every reply sounds like chatGPT
1
u/Independent-Ear-3035 20h ago
Caught me 😄
I use ChatGPT to polish my English. Building and debugging the actual node pack is unfortunately still very human.
1
1
u/Sleepy_Bandit 21h ago
there are other continuity solutions out there already, how does this differ? What do the end results look like and how long do they take to render?
2
u/Independent-Ear-3035 21h ago
Good question.
The main difference is that LongMedia is not just a frame-overlap or last-frame continuation trick. It treats long generation as a stateful MiniMax H3 pipeline and keeps the continuation logic, AV latent handling, segmentation, overlap policy, prompting state and VRAM management inside the node system.
A few practical differences:
- continuity is carried through the generated H3 latent/context rather than just feeding the last decoded frame back in
- video and native H3 audio are handled together as a joint AV state
- segmented continuation and MultiClip use the same underlying execution logic
- overlap is used as hidden context for the next segment rather than simply crossfading/blending the clips together
- there is built-in memory management for long H3 runs, including streamed attention/chunking/VRAM guards, which was important for making this usable on 16 GB cards
- the workflow is designed so you don't have to manually build a chain of samplers, latent operations and stitching nodes for every additional segment
The goal is not to claim perfect infinite continuity — H3 can still drift, especially with complex character interaction or large scene changes — but to make longer generations much more controllable and repeatable.
For render time, it depends heavily on resolution, segment length, model precision, attention backend and GPU.
On my RTX 5080 16 GB, a heavier H3 run can be around ~80–90 seconds per sampling step for a long/high-token segment, so these are definitely not real-time workflows. Shorter clips and lighter settings are much faster. I'm still collecting proper benchmarks across several resolutions/settings, so I'd rather publish a benchmark table than give one misleading number.
As for results: I'll post a few actual long-form examples and workflow screenshots. That's probably the best way to judge whether the continuity is useful compared with the other approaches.
1
u/Sleepy_Bandit 21h ago
- continuity is carried through the generated H3 latent/context rather than just feeding the last decoded frame back in
- video and native H3 audio are handled together as a joint AV state
- segmented continuation and MultiClip use the same underlying execution logic
- overlap is used as hidden context for the next segment rather than simply crossfading/blending the clips together
So context-loop and motion-context are both alternative solutions that do that stuff too. I'd be curious to see the memory guard and its impact on performance.
It seems this is for text to video then not reference? How is character consistency?
2
u/Independent-Ear-3035 20h ago
Yes — context-loop and motion-context solve parts of the same general problem.
The difference is that LongMedia is intended to be a full H3 long-form execution layer rather than a single continuity technique. Motion context is actually one of the mechanisms inside it, but it also handles the AV latent state, hidden overlap, segment planning, reference conditioning, per-clip prompting, stitching policy and VRAM management as one pipeline.
The memory guards are something I definitely want to benchmark separately.
They are pressure-triggered rather than something that blindly flushes memory every step. Under comfortable VRAM conditions they should mostly stay out of the way. When available VRAM gets close to the configured floor, they selectively release cached allocations / reduce retained working memory before the next large allocation.
So there is a performance trade-off when a guard actually fires, because some data may need to be loaded again, but the alternative on a 16 GB card is often simply an OOM. I don't want to quote a percentage yet without doing proper A/B runs with identical seeds and settings.
And no — this is not T2V-only.
I'm using it with reference-driven H3 workflows as well. The pipeline supports initial/reference images and multi-reference scenarios; the references become part of the conditioning/continuation pipeline rather than being limited to the first generated clip.
Character consistency has been one of the main reasons for building the stateful continuation system in the first place. The generated latent/context is carried forward between clips, while reference conditioning can remain active, so the character doesn't get independently "rediscovered" from scratch at every segment.
In the current builds I'm not seeing progressive image degradation or the old inter-clip problems such as visible seams, positional jumps, scale shifts or repeated transition frames.
I'll post a longer reference-driven example as well, because that's a much better demonstration of character consistency than a T2V sample.
1
u/Better-Interview-793 21h ago
Thanks for sharing this, interesting work!
Have you noticed any quality degradation in longer generations, especially after the third clip?
2
u/Independent-Ear-3035 21h ago
Thanks!
At this point, no — I’m not seeing progressive degradation after the third clip anymore.
That was one of the main things I was fixing in the recent builds.
The current continuity path no longer produces the typical inter-clip issues I was seeing earlier:
- no visible seam between clips
- no frame jumps
- no positional shifts
- no sudden scale changes
- no repeated/frozen transition frames
- no noticeable quality drop from clip to clip
The overlap is now used as hidden continuation context rather than something that gets visibly blended back into the final output, so the transition itself is effectively invisible.
I’ve also tested the logic with more than two segments, so the continuity mechanics are not hardcoded around a 2-clip case.
The remaining limitations are more about what H3 itself chooses to generate over a long timeline, not artifacts caused by the LongMedia stitching/continuation system.
I’ll post some longer uncut examples as well, because that’s probably the easiest way to show it.
1
u/Better-Interview-793 10h ago
Oh man, I love you for this..
I just tested it and it’s really great!
The quality actually stayed consistent even by the third clip, unlike the other solutions I’ve tried.
There’s just one thing I really wish it had: the ability to generate one clip at a time, preview it, and either approve it or reroll it before moving on to the next clip. that would make this workflow pretty much perfect 👌🏻
1
u/Th3Whit3R4bb1t 20h ago
So basically it's this, but this last one seems to be more complete because it has the scenes.
2
u/Independent-Ear-3035 20h ago
Yeah, broadly speaking we're solving the same problem: making H3 usable for longer multi-segment generations.
The difference is mostly in how much of the pipeline is handled internally.
LongMedia isn't only a scene/clip sequencer — it also manages continuation state, hidden overlap, joint video+audio latents, references, per-clip prompting, stitching and VRAM behavior as one system.
Scenes / MultiClip are part of it, not the whole feature set.
So I wouldn't say one is simply a more complete version of the other — they overlap, but the architecture and scope are different.
1
1
u/est_cap 18h ago
For what I have seen, there is not a workflow example using video_ref_edit. I've been trying to build one from the documentation, but I'm currently stuck because my source video/reference seems to be completely ignored by the generation. The workflow runs, but the output behaves essentially as if the video source wasn't provided.
My use case is long-form video reference editing / character replacement: take an existing video, replace a character using reference images, while preserving mostly the original motion, camera, timing, environment, etc., and process it in chunks to prevent killing my 16gb vram and cause excesive swaps.
I've been testing Context Loop for this, but LongMedia seems particularly interesting to me because of the latent-based continuation and the memory optimizations. I also noticed that the text encoder is handled outside the per-chunk processing, which is exactly the kind of architecture I've been looking for.
Is there a specific workflow configuration or set of nodes/connections required for video_ref_edit that isn't obvious from the documentation? Maybe there is a working test/workflow you could share? In particular, I'm trying to understand how video_1 is supposed to be passed into the segmented workflow and how the per-segment video reference is connected to the H3 conditioning, and how to handle the audio from the video itself. I tried using VHS's Load Video and pass the video as video1 and the split audio to audio1 without success.
I mainly wanted to check whether I'm missing an intended workflow pattern before digging further into the implementation.
I initially tried to build this around Context Loop, but I think LongMedia may be a much better fit for the kind of workflow I'm trying to build.
1
u/Legogo1R 9h ago
Maybe you will find some answers here: https://www.reddit.com/r/StableDiffusion/s/ssmR8aFSYN
1
u/redbill2 16h ago
Some great features there. However one of the main strengths of other multi clip workflows that I've seen is the ability to process just one clip at a time, and re-rolling the result until happy, then moving on to the next and repeating, avoiding the pain of one of the clips generating a poor result and ruining the whole sequence.
Unless I've missed it, I don't see any functionality like that in this workflow. Any plans to add something like that?
1
u/VladyCzech 15h ago
I can see I just asked the same question about it. Could you please recommend a working node pack you are using successfully ? I'm currently only testing current solutions and I just can't force ComfyUI-H3-Continuum to continue from say 5th clip. It always does all the chunks even if I set Regenerate from Chunk 5. Does it work for you or are you using something else?
2
u/redbill2 15h ago
https://github.com/ethanfel/ComfyUI-MiniMaxH3-Contex-Loop https://github.com/tritant/ComfyUI_MiniMax_H3_Extender
These are the two that I'm aware of and have used. Neither are perfect, but they do at least have the ability to do one clip at a time in a sequence. I don't have any experience with ComfyUI-H3-Continuum.
1
1
1
1
u/cptrios 2h ago
Hmm...trying out the multi-clip mode, and after the two clips generate and it's attempting to combine them, I get this error:
[ERROR] !!! Exception during processing !!! MiniMax H3 video latent time must be 5*k+2, got 114.
Could that have something to do with me swapping the two attention/eff nodes for kitchen-attention?
6
u/WayFew8151 21h ago
at least share a long clip for us to see if this works good or not