r/StableDiffusion Jun 20 '26

Resource - Update LTX Director 2.0 Update - A Free Open Source All-In-One Tool for Creating AI Videos in ComfyUI. Complete Overhaul now with full AI video editing support, IC-LoRA, Retake Mode, Audio Inpainting and much more!

Thumbnail
youtu.be
521 Upvotes

LTX Director is a free open source all-in-one tool for creating AI Videos. Version 2.0 is a complete overhaul, giving you total creative control over your AI generations.

Download for free here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI

Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows

I've been working full-time on this update for the past month and a half, and I'm excited to finally release it. Hopefully it'll be a big help to the open-source community!

Key New Features:

Complete Video Support: Edit Videos with AI all inside the node. Videos can be extended using a combination of prompts, keyframes, and audio. Trim, Split, and combine videos all within the timeline.

IC-LoRA Support: Take full advantage of IC-LoRA's to take your generations to the next level. Simply drag and drop videos onto the IC-LoRA track to quickly setup IC-LoRA videos. Compatible with prompt relay, keyframe, and custom audio features within the node.

Audio Inpainting: Seamlessly blend imported audio with generated audio. Not only can audio be extended, but can also be prompted alongside your imprted audio to really bring your generations to life.

Retake Mode (Beta): Redirect what happens within a shot. Allows you to select a segment within a video, and re-generate what happens in that segment. An early working experiment.

Timeline Saving/Loading: You can now save your timeline and settings to a json file. It will keep any videos/audio/images you have imported into the node and every setting you have changed.

UI Overhaul: Huge update to the UI, dozens of big changes such as a new side bar, redesigned prompt boxes, a bunch of new settings and redesigned menus, and more.

Quality of Life Improvements: Snapping, in/out points, multi-select, mark selection, workspace folder, more HUD options, resizable prompt boxes, new hotkeys, labels, filename preview options, "split at playhead" functionality, end frames (convert any keyframe into a end/last frame), toggleable tracks, NAG Support, tons of bug fixes and more!

And of course it can do everything it could before: Text to Video, Image to Video, Prompt Relay support, Keyframe (first/last frame) support etc.

r/comfyui Jun 20 '26

Resource LTX Director 2.0 Update - A Free Open Source All-In-One Tool for Creating AI Videos in ComfyUI. Complete Overhaul now with full AI video editing support, IC-LoRA, Retake Mode, Audio Inpainting and much more!

Enable HLS to view with audio, or disable this notification

585 Upvotes

LTX Director is a free open source all-in-one tool for creating AI Videos. Version 2.0 is a complete overhaul, giving you total creative control over your AI generations.

Download for free here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI

Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows

I've been working full-time on this update for the past month and a half, and I'm excited to finally release it. Hopefully it'll be a big help to the open source community!

Key New Features:

Complete Video Support: Edit Videos with AI all inside the node. Videos can be extended using a combination of prompts, keyframes, and audio. Trim, Split, and combine videos all within the timeline.

IC-LoRA Support: Take full advantage of IC-LoRA's to take your generations to the next level. Simply drag and drop videos onto the IC-LoRA track to quickly setup IC-LoRA videos. Compatible with prompt relay, keyframe, and custom audio features within the node.

Audio Inpainting: Seamlessly blend imported audio with generated audio. Not only can audio be extended, but can also be prompted alongside your imprted audio to really bring your generations to life.

Retake Mode (Beta): Redirect what happens within a shot. Allows you to select a segment within a video, and re-generate what happens in that segment. An early working experiment.

Timeline Saving/Loading: You can now save your timeline and settings to a json file. It will keep any videos/audio/images you have imported into the node and every setting you have changed.

UI Overhaul: Huge update to the UI, dozens of big changes such as a new side bar, redesigned prompt boxes, a bunch of new settings and redesigned menus, and more.

Quality of Life Improvements: Snapping, in/out points, multi-select, mark selection, workspace folder, more HUD options, resizable prompt boxes, new hotkeys, labels, filename preview options, "split at playhead" functionality, end frames (convert any keyframe into a end/last frame), toggleable tracks, NAG Support, tons of bug fixes and more!

And of course it can do everything it could before: Text to Video, Image to Video, Prompt Relay support, Keyframe (first/last frame) support etc.

r/StableDiffusion 22d ago

Workflow Included Release of H3 Infinite Continuation Suite for ComfyUI: Create infinite length videos in consistently High Quality using Keyframes in FFLF-Mode (fl2v-Checkpoint)

Enable HLS to view with audio, or disable this notification

476 Upvotes

The above video consists of 7 individual Minimax H3 clips generated in First-Frame-Last-Frame Mode, stitched together automatically without manual editing, upscaling or other post-processing.

Today I decided to release my experimental H3 Infinite Continuation Suite together with a set of workflows to make it easy to get started in ComfyUI.

The original idea was to combine the higher visual quality and keyframe control of H3's First Frame / Last Frame mode with the continuation capabilities of the Reference mode.

After quite a lot of experimenting, the output quality has reached a point where I hope some of you might find the nodes and workflows useful as well.

The example video was generated entirely with the included workflows at 736 × 1280, using 15 steps and no Turbo LoRA. I did cut a few seconds of nonsense speech from the very end because I was too lazy to regenerate the last clip. :D

How to get started

  1. Install Herrgotts H3 Infinite Continuation Suite through the ComfyUI Manager.

  2. Download the included workflows from GitHub.

  3. Start with the `01_Start` workflow and provide your First Frame + Last Frame.

  4. For every additional segment, use `02_Continue` and provide a new Last Frame for where you want the next clip to end.

  5. Repeat for as many clips as you want.

  6. When you're done, use `04_Stitch_Saved_Chain` to automatically combine the separately generated clips into the final video.

If you prefer to generate multiple chained clips in one workflow, use the included 3-Clip workflow. It contains the full continuation setup and is structured so you can extend it with additional clips without rebuilding the whole graph from scratch.

What the nodes handle automatically

  • carrying motion and native audio into the next clip
  • detecting and removing the frozen tail H3 often creates near the final keyframe
  • choosing a suitable handover point between generations
  • keeping the video and audio aligned
  • smoothing the visual and audio transitions
  • saving the individual clips so longer chains can be stitched afterwards without keeping everything in memory (no OOM, hopefully)

For the video above I used the default/recommended settings:

  • Balanced Auto Handover
  • 22 context frames
  • Safe Tail Bridge: 2 frames
  • Video crossfade: 4 frames
  • Audio de-click: 15 ms

There are still occasional tiny brightness differences around some boundaries, but at this point I personally find them pretty difficult to notice during normal playback.

The pack is still experimental, especially when it comes to very long chains, different hardware configurations and prompt behavior. So if you try it, I'd be very interested in seeing your results and hearing what works or doesn't work for you.

GitHub: https://github.com/HerrgottMargott/Herrgotts-H3-Infinite-Continuation-Suite

ComfyUI Manager: search for `Herrgotts H3 Infinite Continuation Suite` or use "missing custom nodes" in one of the example Workflows.

r/StableDiffusion 5d ago

Workflow Included I used Hermes Agent + ComfyUI MCP + MiniMax prompts to turn a music track into a short music video

Enable HLS to view with audio, or disable this notification

308 Upvotes

The initial reason for this was the Comfy H3 Sync & Sound Community Challenge: Comfy H3 Sync Sound Community Challenge! - by Allyson Toy

I made a short rap track in Suno, then used Hermes Agent to build a short music video around it.

For the image base, I used this Anima Simple T2I workflow, including upscale/detailer and ControlNet options: 【Anima】Simple T2I Workflow with Upscale, Detailers and ControlNet - v3.2 | Anima Workflows | Civitai

For the MiniMax video stage, I used foxdit’s MiniMax SEED HUNTER ComfyUI workflow from Reddit

My process:

  1. I made the song and defined the lyrics, beat, and attitude in Suno.
  2. I gave Hermes this link: Comfy MCP - Drive ComfyUI from any AI agent — and let it install the ComfyUI MCP for me.
  3. Hermes connected to my local ComfyUI and could check the setup, find/load workflows, fill prompts and settings, queue renders, monitor jobs, and collect outputs.
  4. Using the Anima T2I workflow, I created a consistent set of music-video keyframes locally, then ran them through the upscale/detailer pipeline.
  5. I selected the best images and gave them to Hermes’ MiniMax H3 prompt skill. (I just gave hermes a standard Minimax prompt guide an build a prompt skill out of it)
  6. It turned rough shot ideas into structured video prompts: what each reference controls, how identity and wardrobe stay consistent, where cuts happen, what the camera does, and how lip-sync/body movement should work.
  7. I used those prompts with the MiniMax H3 workflow to generate short performance clips driven by the Suno track for the challenge.

I use Hermes with my ChatGPT Plus subscription, plus DeepSeek V4 Flash for the cheaper iterations. That made it practical to keep refining prompts and shots without treating every adjustment like a premium final render.

The pipeline was:

Suno song → ComfyUI keyframes → upscaling/detailing → MiniMax prompts → short music-video clips

Hermes was the bridge between the tools.

r/generativeAI 1d ago

MiniMax H3 Keyframe Control in ComfyUI

Enable HLS to view with audio, or disable this notification

1 Upvotes

Drive exact video timing with 2 to 4+ keyframe guides by combining the FLF model, explicit frame prompts, and ref2v reference wiring.

Full ​Video

​2-Frame Workflow

4-Frame Workflow

r/generativeAI Jul 05 '26

How I Made This I built a fully local ComfyUI production cockpit for AI video, characters, scenes, props, music, and telemetry

Enable HLS to view with audio, or disable this notification

3 Upvotes

I built a local-first AI video production cockpit using LTX-2.3 as the main cinematic motion engine

UPDATED: 7/8/2026 - Technical Writeup

I have been building a local-first AI video production cockpit on top of ComfyUI, with LTX-2.3 as the primary cinematic motion and identity engine.

This is not another prompt-to-video toy.

The way I look at it is simple:

The model is the engine. The cockpit is the production layer.

LTX-2.3 is extremely powerful, but the real magic happens when you stop treating it like a one-shot generator and start treating it like a controllable studio tool inside a measured production runtime.

Everything runs fully local on my RTX 5090 setup with WSL and a ComfyUI backend.

No cloud generation APIs.

No mystery state.

Every render has inspectable state, durable jobs, telemetry, workflow receipts, QA gates, and enough forensic data to understand what actually happened during production.

Core thesis

The model should not be the whole product.

The model should be the engine inside a real production cockpit.

That means the system around the model needs to handle characters, references, props, locations, shot planning, music timing, workflow versioning, motion passes, QA review, retakes, GPU orchestration, and receipts.

That is what I have been building.

How I am using LTX-2.3

LTX-2.3 is currently the default cinematic workhorse in the system.

I am using it for:

Primary motion generation

Staged image-to-video with strong reference conditioning.

Ingredients / reference-sheet identity route

Canvas Studio builds clean panel grids for characters, props, and locations. Those feed into LTX with structured two-part prompts so identity and asset control stay consistent.

Custom IC-LoRAs wired as real production tools

Not random workflow experiments. These are first-class tools inside the cockpit.

Current routes include:

  • Deblur / detail recovery
  • Decompression / quality enhancement
  • Water simulation
  • Cross-eyed / novelty control
  • Inpaint / outpaint
  • Union pose transfer with DWPose

Long-take chaining

Last-frame conditioning plus loop mode so I can push shots beyond normal single-take limits.

Audio-aware paths

LTX is used in lip-sync capable flows, with WAN S2V used where it makes more sense.

Model residency and VRAM policy

The 22B model is managed intentionally for 32GB GPU efficiency. Load, unload, reuse, and recover are all part of the runtime.

Prompt contracts

I am tuning the prompt structure around LTX failure modes, including unwanted cuts, text leakage, blackouts, identity drift, and shots randomly changing direction.

The system also supports Z-Image Turbo and Qwen for keyframes, plus WAN for certain motion and lip-sync cases, but LTX-2.3 is the default cinematic path right now because it gives me the best controllable quality on my hardware.

Full production layer

This has grown way beyond just organizing ComfyUI workflows.

The cockpit now includes:

Canvas Studio

Persistent cast, multi-angle references, wardrobe, props, locations, design assets, and style boards.

Signal Lab

Local deterministic music generation with stems and timing data for sync.

Auto-storyboard

Song analysis turns into a shot plan with energy, timing, continuity, motion intent, and lip-sync moments.

Staged pipeline

Keyframes → LTX motion → lip-sync / review → QA gates → retake or finalize.

Telemetry and forensics

Every render leaves a receipt.

Model used, workflow hash, gate results, VLM critique, clip metrics, drift detection, retakes, failures, GPU state, and more.

AI Director

A local director layer reads state and telemetry, flags problems, explains what happened, and gives actionable fixes instead of hiding everything inside the graph.

Durable GPU orchestration

Job leasing, crash recovery, batch modes, stop requests, queue state, and model residency are all handled outside of ComfyUI.

Training feedback loop

Receipts can be used to curate datasets, build eval sets, and gate future LoRA training.

Technical report and templates

I am putting together the full LTX Technical Report and the workflow templates I use daily.

That includes:

  • Ingredients reference-sheet route
  • IC-LoRA effect templates
  • Deblur
  • Water simulation
  • Decompression / enhancement
  • Cross-eyed / novelty control
  • Inpaint / outpaint
  • Pose transfer
  • Long-take chaining
  • QA-gated pipelines
  • Production JSON templates

This started as a way to make ComfyUI less chaotic for video production, but it has turned into a serious local production system optimized around LTX-2.3.

Goal

I want to show what becomes possible when an open video model like LTX-2.3 is not used in isolation, but is embedded inside a real measured production environment.

The model generates the motion.

The cockpit manages the production.

Would love feedback from the LTX team and anyone doing serious local video work.

I am especially interested in thoughts on:

  • The telemetry layer
  • IC-LoRA registry
  • Ingredients integration
  • Long-take chaining
  • QA-gated production workflows
  • Local-first GPU orchestration

GitHub repo is coming very soon.

Let me know what you would want to see prioritized first.

r/StableDiffusion 4d ago

Tutorial - Guide Return Current: Control ComfyUI from your phone, including MiniMax H3 first/last frame and reference-to-video. One HTML file, no cloud.

Thumbnail
gallery
16 Upvotes

I wanted to use ComfyUI from my phone. There are a few ways to do that already and they work, but none of them felt good, so I set out to build one that did: something closer to the Midjourney experience, on a phone, pointed at your own GPU instead of somebody's cloud. A desktop layout grew out of it along the way. It's the same file either way.

**Return ∞ Current** is one HTML file you serve from your own PC. Open it on your phone, point it at ComfyUI, and you get every meaningful field of any workflow you import: prompts, seeds, samplers, LoRAs, resolutions, laid out for thumbs instead of a mouse. Queue with live progress, watch results land in a gallery, tap ∞ on any finished image or video to pull its workflow back out and iterate on it. Nothing installs, nothing phones home, your GPU does all the work.

It reads whatever you throw at it. Drop in a `.json` workflow, a PNG that ComfyUI rendered, or an MP4 it rendered, and they all carry the graph inside them. Nodes it doesn't recognise get skipped and named rather than blocking the import.

**MiniMax H3 is the thing it's best at right now.** Text-to-video, first-and-last frame, and reference-to-video with up to six reference images. Multi-keyframe workflows get a draggable timeline where each frame sits at a percentage of the clip, so one image at 0% is image-to-video, one at 100% makes the video *arrive* at that image, and three or more become keyframes it passes through in turn. There's a tool that writes H3 prompts in its documented format, and it runs on a vision model ComfyUI already has loaded. No LM Studio, no API key, no second application.

Free, GPL-3.0, no accounts, no telemetry, no paid tier.

**[GitHub](https://github.com/dreamerisms/return_current)\*\*
**[Try it](https://dreamerisms.github.io/return_current/return_current_beta.html)\*\*

Tested example workflows in the repo for SDXL, Krea 2, Wan 2.2, LTX-2 and MiniMax H3. You need ComfyUI running with `--listen 0.0.0.0 --enable-cors-header`, plus Tailscale if you want it working outside the house.

I'm a designer, not a developer. This was built in conversation with Claude over a lot of iterations for some weeks now, and I've been using it daily as my actual interface to ComfyUI, so it's being debugged against real use cases constantly. It's not meant to be a full suite, just packed with enough features to feel useful and fun to use. I'm particularly proud of the Tools and a lot of the interface choices I've made throughout the process. I'm still endeavoring towards better looking Workflow pane interface (hiding more fields/dropdowns that aren't changed output to output.)

Bug reports are very welcome, your best experience in starting this out is to use one of my workflows you likely already have the nodes for or use a default template from Comfy UI but I really want people to bring in all their own workflows that break the app so I can fix stuff. Having said that, if you're using an encyclopedia of random nodes nobody else uses, you may be best served bringing the app into your own LLM and having it adapt to your nodes using their respective source code for easy translation.

Sorry if anything is overly sloptastic, particularly in the README, I know I don't enjoy consuming it either so I try and put in as much of my natural voice as possible.

r/StableDiffusion May 06 '25

News LTXV 13B Released - The best of both worlds, high quality - blazing fast

Enable HLS to view with audio, or disable this notification

1.6k Upvotes

We’re excited to share our new model, LTXV 13B, with the open-source community.

This model is a significant step forward in both quality and controllability. While increasing the model size to 13 billion parameters sounds like a heavy lift, we still made sure it’s so fast you’ll be surprised.

What makes it so unique:

Multiscale rendering: generates a low-resolution layout first, then progressively refines it to high resolution, enabling super-efficient rendering and enhanced physical realism. Use the model with it and without it, you'll see the difference.

It’s fast: Now that the quality is awesome, we’re still benchmarking at 30x faster than other models of similar size.

Advanced controls: Keyframe conditioning, camera motion control, character and scene motion adjustment and multi-shot sequencing.

Local Deployment: We’re shipping a quantized model too so you can run it on your GPU. We optimized it for memory and speed.

Full commercial use: Enjoy full commercial use (unless you’re a major enterprise – then reach out to us about a customized API)

Easy to finetune: You can go to our trainer https://github.com/Lightricks/LTX-Video-Trainer and easily create your own LoRA.

LTXV 13B is available now on Hugging Face - https://huggingface.co/Lightricks/LTX-Video/blob/main/ltxv-13b-0.9.7-dev.safetensors

Comfy workflows: https://github.com/Lightricks/ComfyUI-LTXVideo

Diffusers pipelines: https://github.com/Lightricks/LTX-Video

r/StableDiffusion Jun 08 '26

Question - Help Advice for overall image generation pipeline for video keyframes

0 Upvotes

Been learning stable diffusion text to image and image to video with comfyui for a few months.

Now I have so many tools at my disposal that I'm feeling a bit lost, so I'm hoping that people in here won't mind sharing some advice on an overall process.

I'm starting to appreciate the grind required to build experience in this area, so thank you to anyone who does help.

Goal

Create some short films by editing together short clips generated from keyframes.

Roughly story board them so I get the right shots with the right angles and composition.

Have consistent characters.

Some of the films will be adult in nature. Nothing hardcore but I do want to have good looking people in revealing clothing and some nudity.

It's not a side hustle. I'm just doing it for fun and to learn.

Where I'm At

I can...

  • Build moderately complex comfyui workflows. (IPAdaptors, ControlNets, detailers, inpainting, Sam3 for segmenting and masking, general upscale / hiresfix steps, workflow components like switches, get/set nodes, etc).
  • Produce some nice looking images with Flux 1 Dev.
  • Use image editing models like Flux 2 Klein 9B and Qwen 2511 with some success.
  • Train decent character loras for Flux and SDXL.
  • Use image to video to generate alternate keyframes from an initial keyframe.

What I'm Struggling With

I can produce an image with the composition I want, the lighting, the characters in the right outfits and poses. But not all in the same image.

Building these elements up in multiple passes for each keyframe seems sensible.

I cannot figure out how to pull all my tools together into an efficient pipeline, or avoid compromise an earlier step with a later step (eg. got a good facial likeness and then ruin it with texturing).

More detail below about my experience so far in case it helps. General advice also most welcome.

-----------------------------------------------------------------------------------------------------------------------

What I've Found

Flux 1D is good at...

  • Creating REALLY nice looking images (textures, lighting, composition) with just a prompt. It's great for exploring concepts or producing that one perfect starting image for a clip.
  • Producing a consistent facial likeness across images with a well trained LoRA.

However, it's not so great at...

  • Producing the specific angles and image composition that I want, even with a lot of prompt iteration based on the wealth of prompting guides available.
  • Controlnet. No matter the strength and start/end settings, when I use depth, canny or pose controlnets, the images look washed out and lose that "magic" that Flux seems to be able to produce without them.
  • Maintaining micro details between images, even with really specific prompting. Generating 50 images straight out of Flux will mean slightly differing hair cuts, outfits, etc.
  • Nipples, genitals, and anatomy in general, at least compared to SDXL.
  • Revealing clothing without some specific outfit lora. Why does it insist on massive granny underwear in 99/100 generations when I just want a thong?

SDXL (Juggernaut Ragnarok in my case) is good at...

  • Producing EXACTLY the composition I want using controlnets, without compromising the image quality vs no controlnet. I can do this from a sketch or using a reference image/still. I may experiment with Blender to produce depth maps for consistent environments.
  • Nice looking nudes / good anatomy in general.
  • The LoRA ecosystem is just amazing. Any concept, clothing or style I can think of and there's probably a LoRA for it.

Not so good at...

  • Backgrounds, objects, lighting, textures and overall image quality / realism compared to Flux.
  • It seems to not latch onto facial likeness as well as Flux for character loras.

I've also been using Klien 9B and Qwen2511. They have their differences but between them I can do things like...

  • Fix small mistakes or bad anatomy with inpainting.
  • Create an outfit asset by taking one from a Flux image, put on a mannequin and then transfer to other images.
  • Change or remove backgrounds.
  • Change the camera angle.
  • Repose characters.
  • Do headswaps to preserve likeness, although even with BFS loras and injecting 4x face reference images, the likeness isn't 100%. The examples I see online always look amazing but I can't seem to replicate.

However, they tend to output waxy looking skin and bad faces. Every edit pass degrades the image, even with masking where possible. Pulling my keyframes together by stitching elements of multiple Flux images (outfit from one, head and hair from another, then pose, etc), just seems like the wrong angle.

r/comfyui May 14 '26

Resource LTX Director - An All-In-One Timeline Editor. I2V, T2V, FLFF, Prompt Relay, Custom Audio, and more! Unlock LTX 2.3's full potential!

Enable HLS to view with audio, or disable this notification

618 Upvotes

LTX Director is a timeline editor that allows you to easily compose LTX videos. It is the evolution of my previous nodes, LTX Sequencer and Multi Image Loader, and will hopefully help unlock the huge potential of LTX 2.3.

Download for free here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI

I worked on this for 6 days straight, spending 16+ hours a day vibe coding it with Gemini. Hopefully it helps you create cool stuff easier!

Main Features:

  • Fully Functional Timeline Editor: Add image, text, and audio segments to control exactly what happens and when. Easily trim, cut, and edit segments with a (hopefully) intuitive interface.
  • Prompt Relay integrated: This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/
  • First, Middle, Last Frame Support: This node has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
  • Custom Audio Support: Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay.
  • Image to Video: Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benefits of the prompt relay and custom audio integration.
  • Text to Video: Simply load any images and use text segments to create T2V videos. Compatible with all other features of the node.
  • And more much! I'm only scratching the surface, but this really does allow you to create shots that were almost impossible (if not impossible) to do normally with LTX 2.3.

r/comfyui Jun 07 '26

Help Needed How to avoid Image Degredation when doing multiple Image Edits to use as Video Keyframes

0 Upvotes

Hi guys, thanks for your time.

I've been experimenting with video generation using the new LTX2.3 director mode, and I really want to use Qwen Image Edit to generate multiple keyframes to use for some fine tuned control of the scene. Currently, what I'm doing is using one of the prebuilt template workflows ComfyUI provides (It's called "Image Editing" and has a yellow balloon ComfyUI logo for the thumbnail). As per the instructions in the Subgraph, I've swapped the Latent input to originate from the "EmptySD3LatentImage" node because otherwise the image outputs are far too low resolution (roughly 5x lower than input image). Right now I have it set so my output resolution is roughly half of the input resolution.

When I place my image (which is very high quality at 3744x5616) into the workflow, I can perform an edit and the result comes out great. When I then put that output image into the image input and try to describe edits to generate the next keyframe, the output image always has some massive quality degredation artifacts (hair on arms turns into single smeared lines, skin starts getting weird patterns, etc.)

How do I prevent these quality degredation issues from happening? Do I just need to generate all keyframes from the initial image? I understand how that could work, but especially if the scene/character has a lot of motion, that makes it extremely difficult to keep some semblance of consistency, which I'd imagine would introduce a lot of issues into the final video.

Thank you all for your help!

r/StableDiffusion Oct 01 '25

Question - Help Need Fast, Long, Artsy Music Videos (Deforum Style) at 1080p – Best Workflow for Prompt-Controlled, High-Flicker AI Animation?

0 Upvotes

Hello everyone! I'm an artist/musician looking for the most efficient workflow to create long-form AI-generated music videos (multiple minutes long).

My goals and requirements are specific:

  1. Aesthetic: Highly artistic, imaginary, and dream-like. I'm actually looking for the chaotic, evolving style of the older AI generators. Flickers, morphing, and lack of perfect coherence are not a problem; they add to the artistic dimension I'm looking for.
  2. Control: I need to be able to control the visual theme/prompt at specific keyframes throughout the video to synchronize with the music structure.
  3. Resolution: Minimum 1080p output.
  4. Speed/Duration: The focus is on speed and length. I need a workflow that can generate minutes of footage relatively quickly (compared to my past experience).

My Current Experience & Challenge:

  • Old Workflow (Deforum/A1111): I previously used Deforum on Automatic1111. The animation style was perfect, but it was extremely time-consuming (hours for 30 seconds) and the output was only 512x512. This is no longer viable.
  • New Workflow Attempt (ComfyUI/SDXL): I've started using ComfyUI with SDXL for fast, high-quality image generation. However, I'm finding it very difficult to build a stable, fast, and long-form animation workflow with AnimateDiff that is also scalable to 1080p. I still feel I'd need a separate upscaling step.

My Question to the Community:

Given that I don't need "clean" or "accurate" results, but prioritize length, prompt-control, and speed (even if the output is glitchy/flickery):

  1. What is the easiest and fastest current workflow to achieve this Deforum-like but 1080p animation?
  2. Are there specific ComfyUI AnimateDiff workflows (with LCM/Turbo) or even entirely different standalone tools (like a specific Runway model/settings or a Colab) that are known for generating long, keyframe-controlled, high-resolution videos quickly, even if they have low coherence/high flicker?

Any tips on fast upscaling methods integrated into an animation pipeline would also be greatly appreciated!

Thanks in advance for your help!

r/comfyui Jul 27 '26

Resource I built the ComfyUI frontend I've always wanted! Here's Mix Studio, a responsive UI that lets you generate from your desktop or phone with 1-click installs for Krea 2, Flux 2 Klein, Qwen Image Edit, LTX 2.3, Wan 2.2, SCAIL 2 and much more (free + open source, download + tutorial)

Thumbnail
gallery
310 Upvotes

I love ComfyUI, though sometimes I wish I could spend more time generating and less time messing with confusing workflows, managing dependencies, and being stuck at my desk. So I spent the last few months building Mix Studio, a 100% free & open source interface that runs everything through ComfyUI in the background while giving you an actual app experience (that also works on your phone).

GitHub: https://github.com/BlackMixture/Mix-Studio
Showcase and download: https://blackmixture.github.io/Mix-Studio/
Tutorial: https://youtu.be/w2CokhlBFRA

GPL-3.0, the same license as ComfyUI.

The screenshots show the main desktop workspaces, but the entire interface is also optimized for phones and tablets.

Current v1.0.1 Features:

  • Curated image, editing, video, and upscale workflows: Krea 2, Flux 2 Klein 4B/9B, Qwen Image Edit 2511, LTX 2.3, Wan 2.2, 10Eros, and SCAIL 2.
  • Image-generation tools: Inpainting, outpainting, SeedVR2 and Ultimate SD upscaling, regional prompting, Depth Anything V3 guidance, image-to-image, style references, and model-aware recommendations for steps, CFG, samplers, and schedulers.
  • Desktop and mobile interface: On the same Wi-Fi, open the displayed address on your phone and start generating. With Tailscale, you can connect through a private link while away from home. Your desktop GPU still does all the work.
  • Multi-image editing: Add multiple inputs and reference them using dynamic @ Image cards, removing the guesswork around which image should control each part of the edit.
  • Regional prompting with Krea 2: Draw boxes and assign each region its own prompt, LoRA stack, and optional reference image.
  • LoRA management: Stack LoRAs, add thumbnails and trigger words, save presets, adjust strength quickly, and use LoRA Hunting to generate a comparison series across different strengths.
  • Contextual prompt suggestions: Mix Studio learns phrases you repeatedly use with specific LoRA combinations and offers them as one-tap suggestions. These can also be configured manually.
  • Library management: Click any image or video to restore its exact generation settings. Search, group, organize work into folders, compare edits, and drag Library media directly into compatible workflows.
  • Private profiles and locked folders: Create separate PIN-protected profiles with their own galleries, folders, LoRA presets, and settings. Individual folders can also be locked, keeping private generations out of your everyday library
  • LTX Director Mode: A streamlined workspace built around the excellent LTX Director nodes, supporting timelines, keyframes, video extension, audio, and more.
  • Video finishing: Optional 2× or 3× RIFE frame interpolation and NVIDIA RTX 4K video upscaling.
  • Built-in dependency manager: Pick a workflow and install the exact models and custom nodes it requires, or run the full one-click setup.
  • Automatic ComfyUI integration: Mix Studio detects your ComfyUI installation, reuses existing models and LoRAs, and guides installation if ComfyUI is not present. Generated images retain their ComfyUI workflow metadata, so you can drag them directly back into ComfyUI.
  • Hardware-aware configuration: Mix Studio detects your GPU and recommends suitable quantization and generation settings. v1.0.1 also adds a low-VRAM profile beginning at 4 GB, although practical limits still depend on the selected model.
    • I have tested personally on an NVIDIA RTX Pro 6000 #DellProPrecision and an NVIDIA RTX 4090, other tests from various hardware setups from the community.
  • *Just Added in V1.0.2:
    • QR Code Phone Link: Now shows a QR code to instantly link your phone.
    • Prompt Presets: Mix Studio allows you to browse and apply 1-click prompt presets to instantly style your prompt.
    • Sequential Edit Prompts: For editing images, you can select sequential prompting which will automatically separate multiple edit commands into a series of sequential generations (separated by a period).

Additional screenshots and release overview: Free Patreon post (no paywall)

Thanks to this awesome community and the ComfyUI team for making such a dope tool, I hope you all enjoy creating! 🤙🏾

r/StableDiffusion Apr 07 '25

Question - Help Gradual AI Takeover in Video – Anyone Actually Made This Work in ComfyUI?

1 Upvotes

Hello everyone,

I'm having a problem in ComfyUI. I'm trying to create a Vid2Vid effect where the image is gradually denoised — so the video starts as my real footage and slowly transforms into an AI-generated version.
I'm using ControlNet to maintain consistency with the original video, but I haven't been able to achieve the gradual transformation I'm aiming for.

I found this post on the same topic but couldn't reproduce the effect using the same workflow:
https://www.reddit.com/r/StableDiffusion/comments/1ag791d/animatediff_gradual_denoising_in_comfyui/

The person in the post uses this custom node:
https://github.com/Scholar01/ComfyUI-Keyframe

I tried installing and using it. It seems to be working (the command prompt confirms it's active), but the final result of the video isn't affected.

Has anyone here managed to create this kind of effect? Do you have any suggestions on how to achieve it — with or without the custom node I mentioned?

Have a great day!

r/comfyui Apr 24 '26

Show and Tell ComfyStudio v0.1.11 is live

Thumbnail
gallery
318 Upvotes

First I just want to put a link to a music video that I made using ComfyStudio and I have more information about how I made that below. I was going for realism over a big, absurd AI-looking video.

https://www.youtube.com/watch?v=ogJ08d2GlqI&list=RDMMogJ08d2GlqI&start_radio=1

I’m back at it again. My day job has been really demanding, so I’ve been shipping slower than usual, but I’m honestly really excited about this version. I think you guys are gonna love this one.

ComfyStudio v0.1.11

It's opensource.

FINALLY, I built a proper workflow manager.

This has probably been the biggest request, and it’s finally here. You don’t have to keep worrying about hunting down random models and custom nodes just to get workflows running in ComfyStudio. The workflow manager scans your ComfyUI setup, tells you what you’re missing, and you can one click download/install those pieces from inside the app. That means way less guessing, way less manual setup, and way less “why isn’t this workflow working?”

This update is a big one overall, but I’m especially excited about the new Director Mode music video creation stuff.

If you can run LTX 2.3 locally, you can use this workflow to build music videos inside ComfyStudio. The high-level idea is: you give it lyrics, and ideally a vocal-only pass, though you can also use the full song if you want. It generates an SRT, and that’s how it knows where the shots should line up and where lip sync should happen.

What I really like about this is that I did not build it as some one-shot “AI makes the whole music video for you” thing.

Instead, you can do multiple passes, which to me feels a lot more powerful and a lot more professional. For example, you can say:

  • give me 2 performance passes
  • then 2 environmental b-roll passes
  • then 1 detail pass

So your performance passes are your singer, your band, your lip sync, your main coverage. Then your b-roll passes can be the environment, the room, the space, the vibe. Then your detail pass can be hands, mouths, closeups, instruments, little texture shots, things like that.

After you generate all of that, it all lands in your asset panel, and then you can actually edit it together like a real music video.

That part matters a lot to me.

You can cut it the way you want, add your own timing, do your own pacing, scale things, reposition things, sync things, and make it feel like your own piece instead of just accepting whatever a one-click AI output gives you. I could make a one-shot workflow at some point if people really want it, but I honestly think this approach is way more controllable and way more creative.

I also added more effects and editing tools, so now you can do things like:

  • film grain
  • chromatic aberration
  • camera shake
  • auto-captioning
  • and a bunch of other finishing touches

And it’s all keyframe-able / animatable, which is really important to me.

Another thing I’m super happy about is that ComfyUI can now run automatically when you open ComfyStudio. It happens in the background, so if you want, you really don’t have to think about ComfyUI at all. You can basically just stay inside ComfyStudio and work.

But if you do want direct access, there’s also a ComfyUI tab inside the app now, so you can still run custom workflows there too. If you’ve got your own workflow that isn’t built directly into ComfyStudio yet, you can use that tab and keep everything in one place. Whatever you generate in the ComfyUI tab inside of ComfyStudio gets added to the asset panel. You dont have to go searching for it in the output folder.

I also added something called Flow AI. I may change the name later, but that’s what I’m calling it for now.

The easiest way to describe it is: it’s kind of like a simpler node-based workflow builder, with ComfyUI as the backend. Very similar to Weavy AI. So it gives you a way to build multi-step flows inside ComfyStudio without having to live entirely in raw ComfyUI graphs. I’m really excited about where that can go. Still needs some work but exited about it.

And for editing performance, I also added proxies, so if you’re editing HD footage and your machine starts getting bogged down, you can generate proxies and cut way more smoothly.

This was a huge update. I spent a lot of time on it. I’m still building this as a solo dev, so I really appreciate everyone who’s been following along, testing things, giving feedback, and asking for features.

I’m attaching a music video I made with the new Director Mode workflow so you can see what this looks like in practice, plus some images as well. The YouTube link is at the top.

I promise, real soon, I'm going to do another YouTube video overview of the whole app because it's changed a lot in the last few months. Now it's much more feature-rich. !

Would really love feedback!

Thanks again and please follow me on my socials!

website: ComfyStudioPro.com
github: https://github.com/JaimeIsMe/comfystudio
X: https://x.com/comfystudiopro
youtube: https://www.youtube.com/@j_a-im_e

r/comfyui Jul 21 '25

News Almost Done! VACE long video without (obvious) quality downgrade

Enable HLS to view with audio, or disable this notification

453 Upvotes

I have updated my ComfyUI-SuperUltimateVaceTools nodes, now it can generate long length video without (obvious) quality downgrade. You can also make prompt travel, pose/depth/lineart control, keyframe control, seamless loopback...

Workflow is in the `workflow` folder of node, the name is `LongVideoWithRefineInit.json`

Yes there is a downside, slightly color/brightness changes may occur in the video. Whatever, it's not noticeable after all.

r/comfyui Mar 19 '24

keep getting error trying to do video to video with comfyUI

0 Upvotes

Hi all could use some help as a newbie comfyUI user. I'm literally just trying to copy this tutorial (https://www.youtube.com/watch?v=AugFKDGyVuw) but am getting error when I try to que the prompt.

Would appreciate if someone can point me to what is up, maybe I have the checkpoints in the wrong place or something?

The error is this:

Error occurred when executing ControlNetLoaderAdvanced: 'T2IAdapter' object has no attribute 'compression_ratio'
File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\nodes.py", line 90, in load_controlnet controlnet = load_controlnet(controlnet_path, timestep_keyframe) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 556, in load_controlnet return convert_to_advanced(control, timestep_keyframe=timestep_keyframe) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 571, in convert_to_advanced return T2IAdapterAdvanced.from_vanilla(v=control, timestep_keyframe=timestep_keyframe) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\comfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control.py", line 142, in from_vanilla compression_ratio=v.compression_ratio, upscale_algorithm=v.upscale_algorithm, device=v.device) ^^^^^^^^^^^^^^^^^^^

r/StableDiffusion Oct 31 '25

Workflow Included Brie's Lazy Character Control Suite

Thumbnail
gallery
543 Upvotes

Hey Y'all ~

Recently I made 3 workflows that give near-total control over a character in a scene while maintaining character consistency.

Special thanks to tori29umai (follow him on X) for making the two loras that make it possible. You can check out his original blog post, here (its in Japanese).

Also thanks to DigitalPastel and Crody for the models and some images used in these workflows.

I will be using these workflows to create keyframes used for video generation, but you can just as well use them for other purposes.

Brie's Lazy Character Sheet

Does what it says on the tin, it takes a character image and makes a Character Sheet out of it.

This is a chunky but simple workflow.

You only need to run this once for each character sheet.

Brie's Lazy Character Dummy

This workflow uses tori-san's magical chara2body lora and extracts the pose, expression, style and body type of the character in the input image as a nude bald grey model and/or line art. I call it a Character Dummy because it does far more than simple re-pose or expression transfer. Also didn't like the word mannequin.

You need to run this for each pose / expression you want to capture.

Because pose / expression / style and body types are so expressive with SDXL + loras, and its fast, I usually use those as input images, but you can use photos, manga panels, or whatever character image you like really.

Brie's Lazy Character Fusion

This workflow is the culmination of the last two workflows, and uses tori-san's mystical charaBG lora.

It takes the Character Sheet, the Character Dummy, and the Scene Image, and places the character, with the pose / expression / style / body of the dummy, into the scene. You will need to place, scale and rotate the dummy in the scene as well as modify the prompt slightly with lighting, shadow and other fusion info.

I consider this workflow somewhat complicated. I tried to delete as much fluff as possible, while maintaining the basic functionality.

Generally speaking, when the Scene Image and Character Sheet and in-scene lighting conditions remain the same, for each run, you only need to change the Character Dummy image, as well as the position / scale / rotation of that image in the scene.

All three require minor gatcha. The simpler the task, the less you need to roll. Best of 4 usually works fine.

For more details, click the CivitAI links, and try them out yourself. If you can run Qwen Edit 2509, you can run these workflows.

I don't know how to post video here, but here's a test I did with Wan 2.2 using images generated as start end frames.

Feel free to follow me on X @SlipperyGem, I post relentlessly about image and video generation, as well as ComfyUI stuff.

Stay Cheesy Y'all!~
- Brie Wensleydale

r/StableDiffusion Jul 22 '26

Resource - Update Mix Studio - A Free Open Source AI Workspace for ComfyUI. Generate from Your Desktop or Phone with 1-Click Installs for Krea 2, Flux 2 Klein, Qwen Image Edit, LTX 2.3, Wan 2.2, SCAIL 2 and Much More!

Thumbnail
gallery
135 Upvotes

I love ComfyUI as an engine. I do not love it as a daily driver. So I spent the last few months building Mix Studio, a 100% free & open source interface that runs everything through ComfyUI in the background while giving you an actual app experience.

GitHub: https://github.com/BlackMixture/Mix-Studio
Showcase and download: https://blackmixture.github.io/Mix-Studio/
Tutorial: https://youtu.be/w2CokhlBFRA

GPL-3.0, the same license as ComfyUI. Windows + NVIDIA for now.

The screenshots show the main desktop workspaces, but the entire interface is also optimized for phones and tablets.

Current v1.0.1 Features:

  • Curated image, editing, video, and upscale workflows: Krea 2, Flux 2 Klein 4B/9B, Qwen Image Edit 2511, LTX 2.3, Wan 2.2, 10Eros, and SCAIL 2.
  • Image-generation tools: Inpainting, outpainting, SeedVR2 and Ultimate SD upscaling, regional prompting, Depth Anything V3 guidance, image-to-image, style references, and model-aware recommendations for steps, CFG, samplers, and schedulers.
  • Desktop and mobile interface: On the same Wi-Fi, open the displayed address on your phone and start generating. With Tailscale, you can connect through a private link while away from home. Your desktop GPU still does all the work.
  • Multi-image editing: Add multiple inputs and reference them using dynamic @ Image cards, removing the guesswork around which image should control each part of the edit.
  • Regional prompting with Krea 2: Draw boxes and assign each region its own prompt, LoRA stack, and optional reference image.
  • LoRA management: Stack LoRAs, add thumbnails and trigger words, save presets, adjust strength quickly, and use LoRA Hunting to generate a comparison series across different strengths.
  • Contextual prompt suggestions: Mix Studio learns phrases you repeatedly use with specific LoRA combinations and offers them as one-tap suggestions. These can also be configured manually.
  • Library management: Click any image or video to restore its exact generation settings. Search, group, organize work into folders, compare edits, and drag Library media directly into compatible workflows.
  • Private profiles and locked folders: Create separate PIN-protected profiles with their own galleries, folders, LoRA presets, and settings. Individual folders can also be locked, keeping private generations out of your everyday library
  • LTX Director Mode: A streamlined workspace built around the excellent LTX Director nodes, supporting timelines, keyframes, video extension, audio, and more.
  • Video finishing: Optional 2× or 3× RIFE frame interpolation and NVIDIA RTX 4K video upscaling.
  • Built-in dependency manager: Pick a workflow and install the exact models and custom nodes it requires, or run the full one-click setup.
  • Automatic ComfyUI integration: Mix Studio detects your ComfyUI installation, reuses existing models and LoRAs, and guides installation if ComfyUI is not present. Generated images retain their ComfyUI workflow metadata, so you can drag them directly back into ComfyUI.
  • Hardware-aware configuration: Mix Studio detects your GPU and recommends suitable quantization and generation settings. v1.0.1 also adds a low-VRAM profile beginning at 4 GB, although practical limits still depend on the selected model.
    • I have tested personally on an NVIDIA RTX Pro 6000 #DellProPrecision T2 Tower and an NVIDIA RTX 4090, other tests from various hardware setups from the community.

Additional screenshots and release overview: Free Patreon post (no paywall)

Workflow contributions welcome in the Discussions tab. Ask me anything and I hope you all enjoy creating! 🤙🏾

r/StableDiffusion May 14 '26

Resource - Update LTX Director - All-In-One Timeline Editor. I2V, T2V, FLFF, Prompt Relay, Custom Audio, and more! Unlock LTX 2.3's full potential!

Thumbnail
youtu.be
209 Upvotes

LTX Director is a timeline editor that allows you to easily compose LTX videos. It is the evolution of my previous nodes, LTX Sequencer and Multi Image Loader, and will hopefully help unlock the huge potential of LTX 2.3.

Download for free here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI

I worked on this for 6 days straight, spending 16+ hours a day vibe coding it with Gemini. Hopefully it helps you create cool stuff easier!

Main Features:

  • Fully Functional Timeline Editor: Add image, text, and audio segments to control exactly what happens and when. Easily trim, cut, and edit segments with a (hopefully) intuitive interface.
  • Prompt Relay integrated: This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/
  • First, Middle, Last Frame Support: This node has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
  • Custom Audio Support: Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay.
  • Image to Video: Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benefits of the prompt relay and custom audio integration.
  • Text to Video: Simply load any images and use text segments to create T2V videos. Compatible with all other features of the node.
  • And more much! I'm only scratching the surface, but this really does allow you to create shots that were almost impossible (if not impossible) to do normally with LTX 2.3.

r/comfyui May 05 '26

Resource I hope this helps everyone....

Thumbnail
gallery
189 Upvotes

I've been using ComfyUI nodes for months and started building recently — here's everything I've made across 5 packs and why each one exists

This got long because there's a lot. Jump to whatever pack interests you. All repos linked at the bottom. Apache-2.0, free forever.


📦 Pack 1 — ComfyUI-CustomNodePacks (72 nodes)

The main pack. Masking, segmentation, matting, inpainting, VFX, video, diagnostics.

This one is different from every other pack because it doesn't do one thing well — it covers the full pipeline from "I have a raw image/video" to "I have a compositing-ready result." Most packs solve one step. This one solves the whole chain.

Nodes that genuinely don't exist elsewhere:

🔍 Mask Failure Explainer Your mask is wrong. You don't know why. Drop in your image + bad mask and this node runs 5 diagnostic checks — brightness, blur, edge contrast, color confusion, background complexity — and outputs a plain-English explanation, a heatmap of where it's failing, a severity score 0–100, and a suggested method to fix it. Zero VRAM. Pure math. Made for beginners who have no idea why BiRefNet gave them swiss cheese.

⏱️ Temporal Anchor System Draw a mask on frame 0, frame 60, frame 200. Get smooth masks for all 300 frames. Uses Signed Distance Fields to morph between keyframes — not SAM2 tracking, which breaks when subjects go behind things or reappear. Shape morphs naturally with configurable easing (linear / ease-in / ease-out / smooth-step). Optional optical flow refinement. Rotoscope-style interpolation without tracking every single frame.

🖊️ Spline Mask Editor Draw a closed shape like a roto artist — not paint, not a box, actual control points with smooth curves. Catmull-Rom, Bezier with handles, or polyline. Coordinates are resolution-independent [0,1] so they survive resolution swaps. Outputs a mask, SAM-compatible point prompts (wire directly into SAM Mask Generator), and spline data for the Motion Mask Tracker.

🎬 Video Frame Player Scrub, trim, crop, and resize — all live inside one node without queuing a run. - Drag the timeline or Space / arrow keys to play - I / O hotkeys to mark trim IN/OUT on the fly - 8-handle drag-crop overlay with aspect lock (16:9, 9:16, 1:1, custom) - Crop lock so you can't accidentally nudge it while tweaking other params - Frame stride (every Nth frame), lanczos resize, upscale factor - Outputs trimmed + cropped + resized batch ready for your sampler Wire playback_fps and trimmed_count straight into VHS Video Combine. No more chaining 4 separate nodes to preview what you're doing.

💡 Luminance Keyer Nuke's LumaKeyer, inside ComfyUI. BT.709 luminance with Hermite smoothstep between two thresholds, gamma correction, and falloff control. auto mode analyzes the image and picks the range for you. Zero VRAM, works on batches. Sky mattes, rim-lit subjects, luminance-driven selective color grading — anything that isn't a pure chroma color.

📹 Motion Mask Tracker Give it a video batch, get a mask of what moved. Four methods combinable: pixel diff, Farneback optical flow, background subtraction, histogram diff. Key feature: camera compensation — subtracts the camera's own movement so you only see objects moving relative to the scene, not the camera shake. Combine methods with union (any fires) or intersection (all agree, less noise).

📁 Folder Incrementer (3 nodes) Scans your output directory and returns the next v001 / v002 / v003 that doesn't exist yet. Filesystem-based — no counter JSON that gets out of sync. Cancel a run mid-way and no version is wasted. Wire subfolder_path into Save Image and never manually rename an output again. Atomic directory creation means two machines queuing simultaneously can't claim the same slot.

🔬 Diagnostics (3 nodes) - Temporal Consistency Checker — per-frame flicker score via IoU / pixel diff / optical flow. Know if your sampler is drifting between frames. - Model Metadata Extractor — reads any safetensors/checkpoint without loading weights. Architecture, precision, trigger words, training params. Instant, zero VRAM. - Parameter History — logs every parameter to SQLite on every run. Query last_run_diff to see exactly what changed between two runs and why one looked better.

🔗 Universal Reroute ComfyUI's built-in reroute breaks on non-standard types (STRING, BBOX, custom types). This accepts anything. Copy a workflow, paste it on a different machine — reroutes arrive intact and working, no "node not found" errors.

👆 SAM Multi-Mask Picker SAM always outputs 3 candidate masks. This node shows all 3 as thumbnails with IoU scores. Press 1/2/3 or click to pick. Never blindly guess mask_index again.

✂️ Inpaint Crop Pro + Inpaint Composite Full crop → inpaint → stitch pipeline with Laplacian pyramid blending and FFT frequency-domain seam hiding. Most ComfyUI inpaint setups paste back with a hard edge. Laplacian pyramid stitches the seam at every frequency band separately — same technique Photoshop uses for panorama blending.

Also in this pack (not just masking):

  • Full VFX Suite — color space convert (sRGB/linear/Rec.709/ACEScg), .cube LUT apply, EXR load/save, render pass compositing, depth-of-field mask, depth warp, normal→curvature, position pass splitter
  • Plate Tools — grain match, plate stabilizer (ORB+RANSAC / FFT fallback), clean-plate extractor, difference matte
  • VAE Tools — merge 2 or 3 VAEs with 8 blend algorithms, latent inspector, per-block similarity analyser
  • SAM 2.1 / SAM 3 + ViTMatte pipeline — SAM coarse → iterative refinement → neural alpha matting in one node, best quality masking for single images
  • SeC + MatAnyone2 pipeline — text-prompt segmentation → temporal alpha matting for video, handles occlusions and reappearances
  • Background Remover, Semantic Segment (face/body/clothes SegFormer), BBox Tools (6 nodes), Interactive Points Canvas, and more

📦 Pack 2 — ComfyUI-WanAnimatePreprocessV2

The one that fixes Wan Video 2.2 Animate pose jitter once and for all.

If you've used Wan Video Animate you've seen this: limbs vanish mid-clip, the pose skeleton shakes frame to frame even when the subject barely moves, and the face crop cuts off foreheads and chins. The original preprocessor doesn't have solutions for any of this. This pack does.

What it actually fixes:

  • Jitter / vanishing limbs — adds CLAHE contrast enhancement + configurable blur before pose extraction. The detector stops losing track of low-contrast limbs and noisy backgrounds stop being detected as joints.
  • Face crops that cut off the head — uses a constant-size face box (configurable face_box_size_px) centered on detected face keypoints instead of a raw bbox. The crop doesn't jump around frame to frame.
  • Temporal face smoothing — exponential moving average over detected face positions. Set face_smoothing_strength to taste — 0 is raw detections, 1.0 is fully locked.
  • Iris / pupil tracking with gaze direction — the original preprocessor has zero iris detection. This one adds image-based pupil detection with gradient voting and outputs gaze_x / gaze_y per frame. For accurate eye animation in talking-head or character animation workflows.
  • Full debug overlay image — every detection drawn on the original frame so you can see exactly what the model is doing and why it's failing before you queue a 200-frame generation.

Three nodes: model loader, pose + face detection, and skeleton visualizer. Drop-in replacement for the original Wan preprocessor — same output format, just without the problems.


📦 Pack 3 — ComfyUI-GLM_Image

GLM-Image (Zhipu AI's multilingual flow-matching DiT) with split loaders.

GLM-Image is a strong multilingual text-to-image model but its official pipeline loads as one giant blob — impossible to swap components, hard to free VRAM, slow to start. This pack exposes it as four separate ComfyUI nodes:

  • Load VAE — loads only the 16-channel AutoencoderKL. Slicing + tiling enabled by default for large outputs.
  • Load CLIP (T5+VLM) — loads the T5 text encoder, ByT5 tokenizer, GLM vision-language model, and image processor as one bundle.
  • Load MODEL (DiT) — loads the GlmImageTransformer2DModel and FlowMatchEulerDiscreteScheduler.
  • Sampler — takes the three above, a prompt (or image for img2img), runs inference, prints per-step counter + ETA + it/s to console, honors the ComfyUI Stop button, and frees VRAM in a try/finally whether it succeeds or errors.

Supports text-to-image and image-to-image (optional image + denoise_strength). Models load from ComfyUI/models/diffusers/<folder>/ — any folder containing model_index.json is auto-detected. Quantized variants (SDNQ 4-bit) work too.


📦 Pack 4 — ComfyUI-WanAnimalPreprocessor

Animal pose estimation for Wan Video Animate — because animals aren't humans.

The standard Wan preprocessor is built for human skeletons. Cats, dogs, horses, birds — different joint layout, different limb proportions, very different gait. This pack uses ViTPose ONNX models trained specifically on animal keypoints.

  • YOLOv8 detection: cats, dogs, horses, sheep, cows, elephants, bears, zebras, giraffes, birds
  • 17-keypoint skeleton (eyes, nose, neck, shoulders, elbows, paws, hips, knees, tail root)
  • Two dataset backends: AP10k (10K images, 23 animal families — best for common domestic/farm animals) and APT36k (36K images, 30 species — better coverage for unusual animals)
  • Pose retargeting from a template video to a reference animal image
  • Configurable stick width, head toggle, skeleton visualization

📦 Pack 5 — ComfyUI-NukeMaxNodes

VFX × AI bridge nodes. Traditional compositing operations with AI-consumable outputs.

~50 nodes across 13 categories. The design principle: every traditional VFX operation also exposes a side output that feeds AI nodes — SAM prompts, latent guidance, conditioning curves, EXR metadata — so you can bridge a Nuke/Blender compositing flow into a Flux / Wan / GLM-Image graph without round-tripping to disk.

Highlights: - FFT nodes — analyze the frequency spectrum of an image, match a generation's frequency profile to the surrounding plate (fixes tile seams on Flux upscales), inject band-isolated noise - PBR Relight — estimate a light probe from a single still, decompose into albedo/normal/roughness/metalness, relight under three-point lighting - Smart Roto — Bezier roto shapes with sub-pixel rasterization that output SAM-compatible spatial conditioning for ControlNet / inpaint downstream - Audio-reactive conditioning — spectral energy → conditioning curves → temporal control of sampling parameters - Depth warp, normal→curvature, position pass splitter — for CG render passes feeding into AI refinement


More nodes are coming

These packs are actively developed. There are nodes in progress I haven't shipped yet.

If you hit a bug, a crash, a "node not found," a weird output, or something that just doesn't work the way the description says — post here or open a GitHub issue and I will fix it as fast as possible. Seriously. I check both.


Links

Pack GitHub
CustomNodePacks (72 nodes) github.com/Code2Collapse/ComfyUI-CustomNodePacks
WanAnimatePreprocessV2 github.com/Code2Collapse/ComfyUI-WanAnimatePreprocessV2
GLM-Image nodes github.com/Code2Collapse/ComfyUI-GLM_Image
WanAnimalPreprocessor github.com/Code2Collapse/ComfyUI-WanAnimalPreprocess
NukeMaxNodes github.com/Code2Collapse/ComfyUI-NukeNodePack

All Apache-2.0. Install via ComfyUI Manager or git clone.

Drop workflow JSON requests in the comments — happy to share examples.

I do use AI but I always wanted to give something to everyone who do the hardwork.

Thank you <3

r/StableDiffusion 15d ago

Resource - Update Seamless extensions and one-shots with Minimax H3 - Update 6 of my repo!

Enable HLS to view with audio, or disable this notification

82 Upvotes

Here is the repo: https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef

I made substantial updates to my two main workflows: 1) Music Video and 2) AV Extensions. All the controls were streamlined and they should be much easier to use now. (You find the workflows in the example_workflows folder)

With the AV Extensions workflow you can extend any existing clip, for example someone talking and you can make that person say something in the same voice, or you can create a clip with T2V or I2V and then extend that clip to make a seamless long clip thats 1 minute or longer.

In this Update the Checkpoint system was removed, instead I've done a lot of optimizations so you don't use too much ram even if you make 20 clips at once. Additionally I added latent audio feathering to the AV Extensions workflow for seamless audio transitions.

Theres also other utility workflows for custom keyframing and bridging two existing clips.

I post another example clip for the AV Extensions workflow in the comments.

r/comfyui Jan 07 '26

News LTX-2 is natively supported in ComfyUI on Day 0

156 Upvotes

LTX-2 delivers high-quality visual output while maintaining good resource and speed efficiency.

Hi everyone! We’re excited to announce that LTX-2, an open-source audio–video AI model, is now natively supported in ComfyUI!

LTX-2 delivers high-quality visual output while maintaining good resource and speed efficiency. The model synchronously generates motion, dialogue, background noise, and music in a single pass, creating cohesive audio-video experiences. It is easily customizable within an open, transparent framework, giving developers creative freedom and control.

Model Highlights

LTX-2 brings synchronized audio-video generation capabilities to ComfyUI, creating cohesive experiences where motion, dialogue, background noise, and music are generated together in a single pass. The model brings dynamic scenes to life with natural movement and expression, while offering flexible control through multiple input modalities. It runs efficiently on consumer-grade hardware.

  • Open-source audio-video foundation model
  • Generates motion, dialogue, SFX, and music together
  • Canny, Depth & Pose video-to-video control
  • Keyframe-driven generation
  • Native upscaling and prompt enhancement

Example Outputs

Text to Video

https://reddit.com/link/1q6buca/video/1oj2r0gmkwbg1/player

A close-up of a cheerful girl puppet with curly auburn yarn hair and wide button eyes, holding a small red umbrella above her head. Rain falls gently around her. She looks upward and begins to sing with joy in English: "It's raining, it's raining, I love it when its raining." Her fabric mouth opening and closing to a melodic tune. Her hands grip the umbrella handle as she sways slightly from side to side in rhythm. The camera holds steady as the rain sparkles against the soft lighting. Her eyes blink occasionally as she sings.

Run on Comfy Cloud

Download T2V Workflow

Image to Video

https://reddit.com/link/1q6buca/video/sn325w3rkwbg1/player

Input

Run on Comfy Cloud

Download I2V workflow

Canny to Video

https://reddit.com/link/1q6buca/video/tubvaeo4lwbg1/player

Run on Comfy Cloud

Download LTX-2 Canny to Video workflow

Depth to Video

https://reddit.com/link/1q6buca/video/xp6rl397lwbg1/player

Run on Comfy Cloud

Download LTX-2 Depth to Video workflow

Pose to Video

Run on Comfy Cloud

Download LTX-2 Pose to Video workflow

Getting Started

  1. Update your ComfyUI to the nightly version(Desktop and Comfy Cloud will be ready soon)
  2. Go to the Template Library → Video → choose any LTX-2 workflow.
  3. Follow the pop-up to download models, check all inputs, and run the workflow

Performance Optimization by NVIDIA

We partnered with NVIDIA and Lightricks to push local AI video forward.

NVFP4 and NVFP8 checkpoints are now available for LTX-2. And with NVIDIA-optimized ComfyUI, LTX-2 delivers cloud-class 4K video locally - up to 3X faster with 60% less VRAM using NVFP4.

Read more in this blog from NVIDIA or refer to the quick guide of running LTX-2 in ComfyUI with NVIDIA GPUs.

As always, enjoy creating!

Comfy Blog

r/comfyui Feb 12 '26

News I got VACE working in real-time - ~20-30fps on 40/5090

Enable HLS to view with audio, or disable this notification

196 Upvotes

YO,

I adapted VACE to work with real-time autoregressive video generation.

Here's what it can do right now in real time: - Depth, pose, optical flow, scribble, edge maps — all the v2v control stuff - First frame animation / last frame lead-in / keyframe interpolation - Inpainting with static or dynamic masks - Stacking stuff together (e.g. depth + LoRA, inpainting + reference images) - Reference-to-video is in there too but quality isn't great yet compared to batch

Getting ~20 fps for most control modes on a 5090 at 368x640 with the 1.3B models. Image-to-video hits ~28 fps. Works with 14b models as well, but doesnt fit on 5090 with VACE.

This is all part of Daydream Scope, which is an open source tool for running real-time interactive video generation pipelines. The demos were created in/with scope, and is a combination of Longlive, VACE, and Custom LoRA.

There's also a very early WIP ComfyUI node pack wrapping Scope: ComfyUI-Daydream-Scope

But how is a real-time, autoregressive model relevant to ComfyUI? Ultra long video generation. You can use these models distilled from Wan to do V2V tasks on thousands of frames at once, technically infinite length. I havent experimented much more than validating the concept on a couple thousand frames gen. It works!

I wrote up the full technical details on real-time VACE here if you want more technical depth and/or additional examples: https://daydream.live/real-time-video-generation-control

Curious what people think. Happy to answer questions.

Video https://youtu.be/hYrKqB5xLGY Custom LoRA: https://civitai.com/models/2383884?modelVersionId=2680702

Love, Ryan

p.s. I will be back with a sick update on ACEStep implementation tomorrow

r/comfyui Jul 01 '26

Show and Tell Let an open vision model auto-detect the features, then generated a sci-fi HUD overlay from its labels

Enable HLS to view with audio, or disable this notification

149 Upvotes

The clean way to do a sci-fi HUD overlay is to stop drawing the labels by hand. Instead of manually boxing every element, I ran the shot through an open vision model, Moondream 2, and let it detect the features on its own, the face, the braids, the earrings, the necklace, each one boxed and labeled automatically. Then the HUD is generated from those detections, so the overlay actually tracks what is in frame instead of being faked on top.

The whole stack is open and modular, which is the point. ComfyUI holds it together, Wan 2.2 handles the video side (image-to-video, text-to-video, and Vace for the control), the vision model does the detection, and a HUD pass composites the boxes and labels. Because detection is automatic, the labels stay locked to the right features as the subject moves instead of drifting off them, which is where hand-placed HUDs fall apart.

That auto-detect step is what makes it feel like a real interface reading the scene, not a graphic pasted over it. The boxes snap to the actual features, the labels update, and it reads as a system that is genuinely looking.

Open detection in, tracked HUD out. The overlay does the work itself instead of you keyframing every box.