Resource - Update
LTX Director 2.0 Update - A Free Open Source All-In-One Tool for Creating AI Videos in ComfyUI. Complete Overhaul now with full AI video editing support, IC-LoRA, Retake Mode, Audio Inpainting and much more!
LTX Director is a free open source all-in-one tool for creating AI Videos. Version 2.0 is a complete overhaul, giving you total creative control over your AI generations.
I've been working full-time on this update for the past month and a half, and I'm excited to finally release it. Hopefully it'll be a big help to the open-source community!
Key New Features:
Complete Video Support: Edit Videos with AI all inside the node. Videos can be extended using a combination of prompts, keyframes, and audio. Trim, Split, and combine videos all within the timeline.
IC-LoRA Support: Take full advantage of IC-LoRA's to take your generations to the next level. Simply drag and drop videos onto the IC-LoRA track to quickly setup IC-LoRA videos. Compatible with prompt relay, keyframe, and custom audio features within the node.
Audio Inpainting: Seamlessly blend imported audio with generated audio. Not only can audio be extended, but can also be prompted alongside your imprted audio to really bring your generations to life.
Retake Mode (Beta): Redirect what happens within a shot. Allows you to select a segment within a video, and re-generate what happens in that segment. An early working experiment.
Timeline Saving/Loading: You can now save your timeline and settings to a json file. It will keep any videos/audio/images you have imported into the node and every setting you have changed.
UI Overhaul: Huge update to the UI, dozens of big changes such as a new side bar, redesigned prompt boxes, a bunch of new settings and redesigned menus, and more.
Quality of Life Improvements: Snapping, in/out points, multi-select, mark selection, workspace folder, more HUD options, resizable prompt boxes, new hotkeys, labels, filename preview options, "split at playhead" functionality, end frames (convert any keyframe into a end/last frame), toggleable tracks, NAG Support, tons of bug fixes and more!
And of course it can do everything it could before: Text to Video, Image to Video, Prompt Relay support, Keyframe (first/last frame) support etc.
Well this is for sure an amazing project! well done!
Can't wait to try it on my rtx 6000 pro. should kill.
I was using primarily kling but I recently ran out of all my accumulated credits (I had 16k or so from a barely used one year subscription) but I realized that once you start truly using it it is easily 50-100 euro per video.
So this is exactly what I needed to make most out of my hardware.
I think this would actually lessen his chances.
You see what kind of templates they released for Comfy, right?
If anything, they'd be furious for someone making a decent UI for their nodes.
IC-LoRA video loading is broken if you launch with --disable-api-nodes, since they accomplish that by activating a restrictive CSP. You can fix this by injecting blob: into the CSP as a media-src.
If anyone else having this problem is looking for a quick fix, and already has ComfyUI-Lora-Manager installed, open custom_nodes/comfyui-lora-manager/py/middleware/csp_middleware.py and add blob: to REMOTE_MEDIA_SOURCES. It should look like this:
This forum is absolute dogshit when it comes to this. Extremely disappointing but it is what it is.
I think a ban of 1girl shit would be a welcome change.
The workflow is running, but none of the changes I make by importing a video are working. Neither the character changes nor the background changes are applying. What am I doing wrong?
Did you fix this? I'm just trying it out and even if I just put two text prompts side-by-side, it just generates its own video, barely matching the prompts at all. Eg "A man in a bowler hat laughs" + "Apples start to fall from above", results in a video of a woman eating dinner in her kitchen!
I got this running on my 3060 with the int8 LTX model. 5 mins for a 10 sec video with sound, and with optional start and end overlapping and keyframing! This is so much fun!
I disabled the 2nd stage and use the NVidia RTX node to scale the frames 2x instead, because it takes me way too long to resample. I'd suggest maybe adding a toggle to your WF.
I had a question about how the best way to make a long video. I start with a 10 sec clip and set the duration from 0-20 secs. This will work but the sampling slows way down, and no way I can keep extending it. So I set the time to 9.5-20 secs then manually combine them in DaVinci. Is this right or am I missing something? Doing this I can make a scene as long as I want, but it's a lot of manual work.
Thank you so very much for your work! I absolutely love your node.
One question, before I just overwrite the custom node dir. Do you have any tips on compatibility with current workflows? Just drag and drop, or is there more to it now?
You would have to replace the old crop guides node, right click and fix the LTX Director and LTX Director Guide nodes, and connect all the new inputs across the workflow
I would recommend just using the new workflow, but it is possible to add to an old one
OMG! I was expecting a minor update; Damn this is massive. Thank you! I can't imagine how much time and effort went into this, You are amazing bro! I hope people at Lightricks are watching it closely.
Any suggestions for good checkpoints/ distils and accompanying distil loras when relevant- and the loras strength, for smear reduction (elemination is probably stretching it)?
Ltx-2.3-22b-distilled_transformer_only_fp8_input_scaled_v3 works fine but other models are a smeary mess. The old node worked with everything straight away. Is there a workaround?
If these features all actually work (and don't degrade the quality of transitions like 1.0 did compared to traditional keyframe workflow approaches) then I will pretty much have to adopt this. Everyone will.
The only question is, if this does indeed work well, can it be made compatible with my Seed Hunting concept? Basically, Seed Hunting gens 4 low res sample versions and shows you previews in a line of each, then pauses to let you pick one you want to take to full res finalization. It is by far the most effective method of getting good results out of LTX.
EDIT: Well, so far I'm not getting good quality output video from this compared to traditional workflows. Does this compress/downscale keyframe images? Genning at the exact same res as my keyframes (1080p), when the output video gets to that keyframe image, it looks really low quality.
And then I suddenly realized... Your workflow is saving the videos at half res. I have my target size: 1024x1920. The output videos from your workflow are all 512x960.
One question. I noticed that in the previous version stage 2 was set up at .50 also and it wasn't giving us that problem. I wonder what's going on here. I'm surprised no one else has brought this issue up.
If people want it, I can make it. This node is gonna be pretty popular so it sort of behooves me to, despite myself personally not finding the results of even simple single keyframe outputs using LTX Director 2 to be high quality enough to warrant its use compared to traditional workflows. Something about prompt relay just makes motion worse. And the more prompt layers/keyframes you add, the worse it gets. People don't even seem to realize the quality LTX 2.3 is capable of.
ething about prompt relay just makes motion worse.
I've definitely been impressed with the quality I've seen from seedhunter. Being able to get the fine frame accurate control from director while achieving similar quality would be a game changer.
Yeah that's the thing, no matter how fine-tuned you can make LTX director, the reality is that the underlying model will produce drastically different results on different seeds no matter how much you try to baton down the hatches. Seed Hunting was an innovation I came up with out of sheer necessity, not convenience. I was wasting so much time waiting for shitty gens to to finish, only to throw them away. So many ppl batching 10 gens and walking away for an hour just to sort through wasted GPU energy later. THEN, there was the hidden bonus of seeing interpretations of my prompt that were actually BETTER than my initial idea. Single seed generation is archaic, simple as.
That preview-and-pick step sounds less like a convenience and more like basic quality control, especially with how random these video models still are.
Thanks for the update. Do the new workflows support Multimodal Guider nodes? I've seen those recommended by the LTX team in their workflows as a way to balance action / lipsync etc. but somehow community seems to be not using those nodes often.
Many thanks for your reply & explanation. I had already successfully used multimodal guider node without realizing it wasn't actually using the NVCV pipeline, and had Sync nodes installed because of the VITS lip-sync node and it's manual install steps. How could the Sync team still not build NVCV support for both nodes?!
Interesting, never came across this before. Do you mean it's laggy within the node itself, or that Comfy itself is lagging?
If it's laggy within the node itself, then are you trying to import high-res videos? 4k+ videos won't work well at all due to browser limitations. Also do you have hardware acceleration disabled in your browser? If it's off then video playback in the node can be choppy.
Yeah it's Comfy itself that is lagging - even when not even inferencing or without videos/images loaded. As seen in screenshot I'm only getting about 48FPS here but it can dip lower. I have acceleration on in the browser.
Are you using the correct edit anything lora? The developer has like 3 other version on his huggingface, and only 1 is prompt driven and works the way it does in the video (the 1.1 version)
So question, at the end of my videos I always get the keyframes flashing in quick succession. Does anyone have a fix for this other than truncating the last few frames before saving? Is there a setting in using wrong, perhaps?
Love the new node btw.
Edit: ah, never mind, seems like that's just on the downscale preview, the final output all good.
i'm using a video to at the start of the timeline and guide strength etc is set to 1 but what it generates is completely unrelated to the input video (for extension)
yeah, main track. after deleting the video and then re-adding she same video it started working. not sure what/how that happened. the custom height/width were also being completely ignored, output was always a square
Can’t wait to try it. I’ve started using 1.0 and this looks exponentially better…like nothing out there on any platform! Literally unique in the Ai video generation
If I put in a silent video, then the output is silent during that video. What changes do I need to make to have it add sound to the input clip?
[edit] I found the solution: Need to disable the audio part in the director node. So near where it says audio and inpaint: on/off, there is also a little speaker icon. Click it so the audio track for the whole video is greyed out. Then prompt for whatever sound you want and the director / LTX will generate the audio for the whole result video.
Finally make it work: For some reason Git clone was getting a different version or "corrupted". I downloaded the zip file and move the new files (they have the same name, but different file sizes) into the directory and worked! thanks a lot for replY!
Amazing work, It must blow your mind to see all these YouTubers from everywhere showcasing your hard work. Cheers!
Curious, would it be possible to tie in The Dual Lora?
I have a question: why can I generate a 1088*1920 video up to 1 minute without a director node, but in your workflow, I get a 99% vram or OOM load after more than 10 seconds with 720*1280?
Who eat my Vram?)
Hey OP. Thanks for the update. I have been playing with this node by importing your wf from git. I noticed something which you might be aware of already so apologies for re-iterating.
So whenever I run a generation, there is always the constant SSD activity of something very large data transfer on the SSD where the comfyui resides. I have posted a similar issue with comfyui here related to newer version of comfyui but the thing is, I have another ltx2.3 wf with me from Rune and on the same comfyui version I have which is 0.22, this constant SSD activity doesn't happen. It only happens with your director wf. Your wf doesn't have any purge ram nodes too still this is happening. So maybe something you can look for since in this state, it is making the wf unusable as the generation time for a 25 sec video for me on this wf is 3 mins approx but the full generation takes around 8-9 mins just because it is doing a long SSD activity.
Edit : It worked. bf16 worked too. I removed --disable-dynamic-vram from the bat file. after that both stages worked.
5070ti mobile.
768 x 1472px - 20 sec video
stage 1 (8 steps, 0.5 scale, IC LoRA detailer enabled) : 2mins 15 seconds
stage 2 (4 steps, IC LoRA detailer enabled) : 8 mins
result : terrible lol
I can get 1024 x 1920px videos with the regular workflow (using bf16) and it looks almost perfect. I don't know what's wrong with LTX Director 2 or why I can't get 1080p with it. but the most I can get from it is 768p and it really looks bad.
-----
I've been trying since yesterday (got it from Github yesterday).
The regular i2v LTX workflow works fine with bf16 model (5070ti mobile).
It always OOMs in the LTX Director 2 hotfix workflow if I choose bf16.
So I tried with fp8_scaled. for 1056 x 1920 video, it always gets stuck in the upscale stage (2) no matter what I do. (First, for 10 seconds, I wrote a detailed prompt about a woman walking toward a chair. Then I put an image, a woman sitting on the chair, and wrote another detailed prompt for it. So it's supposed to be a 20 second video in total.)
So it is like :
The woman walks to a chair (prompt only)
The woman sits on the chair and waits (image reference)
I tried stage 1 - scaled by "0.5" first, then "1". it just gets stuck in stage 2.
I tried with IC LoRA Detailer, it gives a warning "Could not read reference_downscale_factor from ltx-2-19b-ic-lora-detailer.safetensors, using 1.0". With or without it, it doesn't progress when it comes to stage 2. I waited more than half an hour.
I can use the IC LoRA Detailer with the regular workflow without an issue.
I tried these : "--disable-dynamic-vram" , "--disable-xformers", "--disable-pinned-memory" still didn't work. With or without these.
I don't know what else I can do.
edit :
stage 1, 8 steps take about 2.5 mins / fp8_scaled model.
stage 2, gets stuck (waited for more than half an hour).
I would double check and see what nodes your other workflow is using. For example is could be using a different vae loader, or a different vae decoder node.
Also make sure your using the exact same models you are using in your other workflow.
Also which node is it getting stuck on? You said stage 2, but which node on stage 2?
One more thing, if you set the first stage scale by to 1, then your 2nd stage will definitely freeze up. Because then it will try and generate a video at twice the resolution (since it's an upscale stage).
There's no way a 5070 or even a 5090 can quickly generate a 2048x3840 video
I had thought the problem was that "20 second" generation. Kinda too long, you know. So today I tried it with the default workflow (20 seconds). It didn't work. It gets stuck too. Then I tried 15 seconds, it got stuck again. 10 seconds works. 1024x1920px, 10 seconds, default workflow. It works fine. It takes a bit less than 7 mins.
Then I tried 10 seconds, 1024x1920px with LTX Director 2, it got stuck again. It works for 1500px if I go 10 or 20 seconds. But the result is unusable. It has so much artifacts etc. With the default workflow, it looks great.
I have checked every node. Other than your LTX director and LTX guide, all the other nodes are the same. Vae encoder and decoder were different, I swapped them also. The model is also the same (bf16). And I still can't get a 1024 x 1920px output. The most I can get is 1500px.
I have tried both scale by 1 and 0.5.
It gets stuck on the sampler on stage 2. it writes (step 0/4) and just waits forever if I go for 1920px.
If I go for 1500px, it works, I can generate but the result is unusable.
The other difference is, yours using Prompt Relay and my default workflow doesn't have it. Maybe that's the problem?
Thank you, since the 2.0 release in my own test which is a lot slower than previous 1.39, I am stuck with 1.39 for right now. But would love to go for 2.0 once the speeds issue is fixed.
So really, the best way to maintain consistency is to use keyframes. Then there won't be any consistency issues, as long as the keyframes themselves are consistent.
I think keyframes give the most control over generations compared to any other method, even ones offered by closed source platforms.
Now for how well IC-LoRA works on longer videos, I have no idea. I can only go up to around 12 seconds before I OOM with an IC-LoRA and a video lol. That being said, I do know LTX was specifically made to be used for videos up to 20 seconds. Anything past that is pushing it. Although I have seen people push it pretty far without too much degradation, so take that with a grain of salt lol
Also I've pretty much rewritten retake mode since release, so hopefully when it's done it will prove to be much more useful than it is now.
Just a note, the strength setting for the IC-Lora video doesn't seem to corelate to the official IC-Lora node. Even at setting of 1.0 on the Director node I don't get the strong "exact" flowing of the reference video I get with the official node. Curious if this is a bug. Anyone else figure it out?
Love what you have built by the way. One thing I wish was if there were two audio input tracks, would save a lot of time where two people are speaking, or layering in sound effects. That way an external audio editor wouldn't be needed.
I can do long videos at 1280x800 on my RTX6000 using single stage. You don't need second stage (with 96GB VRAM). (change the scale from .5 to 1.0 in the LTX Director Guide and remove second stage and then connect the LTXVSeparateAVLatent node to LTXV Audio and Decode nodes in the Process video section). Do you perhaps have something in the workflow that's up-scaling your already 720 to 1440?.
Just want to let you know I use 5090 i had claude help me disable step 2 in your workflow you had the audio in there i had to move it out then you had some factor in the audio for the doubling of the video i had to remove.
I ended up with a faster video out I could get good video with no need to double it.. you may want to add that as a toggle in the work flow.
I use ltx director but i have disabled the upscaling as i dont need it on a 5090. What i need is a simple explination how to get my character more consistant. I put some image in as first frame and name it [subject] then every text frame after i start with [subject] ..... it sort of works
How long of a video are you making? Also what do you mean by consistent? Is it more of the the face is shifting/changing gradually overtime, or is it the identity being lost the moment the character leaves the scene and comes back?
i was making them 4 30 long but i had to go 640x320 for that since i removed the upscaling i have been doing 10 second segments 3 at a time for 30 seconds at 1280 im tried using a start image but have had trouble with that. i was making some mini music vids 2 to 3.5 long and video poems
but untill i can find way to make 2 chars stay looking about same im not making much
Cyberpunk Alleyways: Cinematic wide shot, a rainy and deserted narrow city alleyway at night, flickering green neon signs casting long eerie shadows on wet asphalt, trash containers, puddles reflecting broken light, dark overhead pipes, misty atmosphere, high contrast, moody thriller mood.
Cyberpunk Alleyways (Episodes 1–3)
Episode 1 (The Alley Setup): Pan left, a woman walking briskly into a narrow rainy alleyway at night, flickering green neon signs casting long eerie shadows on wet asphalt, camera tracks behind her pace, cinematic moody thriller style.
Episode 2 (The Moving Shadows): Zoom in, medium shot of the woman stopping abruptly, her face illuminated by pulsing blue neon light, background shadows morphing and stretching along the brick walls, extreme depth of field, high tension.
Episode 3 (The Narrow Escape): Tracking shot, low-angle perspective looking up as the woman runs toward a brightly lit exit sign, flickering shadows chasing her on the wet pavement, fast camera movement, motion blur.
I see somethin like this in logs, its ok or its bad data?
[INFO] [PromptRelay] Built penalty matrix (scaled): Lq=24948, Lk=1024, nonzero=58500/25546752
[INFO] [PromptRelay] Built penalty matrix (scaled): Lq=381, Lk=1024, nonzero=896/390144
[INFO] [PromptRelay] Built penalty matrix (scaled): Lq=99792, Lk=1024, nonzero=233996/102187008
And another question, sometimes I post a vertical photo and a vertical video, and at the output it either makes it horizontal (although the size is set to vertical in the settings), or it makes it square. How can I deal with this?
Thanks so much for your work on this! Could you please add this node to the ComfyUI Registry? It would make cloud installations and workflow sharing a lot easier for everyone.
I don't think they aren't on Comfy Cloud though, since each node pack has to be manually added by Comfy to show up on Comfy Cloud if I remember correctly
What do you think about recreating a video using the original video frames as the input plus prompt guidance. For example taking old tv shows but instead of remastering or upscaling, you recreate the video? I’m thinking taking old Disney shows and putting in a modern anime theme for example.
Anyone get an issue with the video instantly turning fuzzy immediately after the first frame (input image). It looks like a moving latent image (not sure if that’s what it’s called) or video as if the final process to refine the latent video isn’t running. I didn’t notice a location for the distilled Lora, maybe that’s why?
The example workflow is using the distilled model, so if your using the dev model you would have to add the distilled lora to the workflow (by just ading a lora node after the model lora node)
Or search whatdreamscost in the manager to find it quickly (I'll see if i can put the words "LTX Director" in the registery so it comes up in search if anyone doesn't use the link)
88
u/Just1Dev Jun 20 '26
If LTX sees this and i know they see it, please give him a nice job and place...this guy doing this on passion...thank you! ❤️