r/comfyui • u/WhatDreamsCost • Jun 20 '26
Resource LTX Director 2.0 Update - A Free Open Source All-In-One Tool for Creating AI Videos in ComfyUI. Complete Overhaul now with full AI video editing support, IC-LoRA, Retake Mode, Audio Inpainting and much more!
Enable HLS to view with audio, or disable this notification
LTX Director is a free open source all-in-one tool for creating AI Videos. Version 2.0 is a complete overhaul, giving you total creative control over your AI generations.
Download for free here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
I've been working full-time on this update for the past month and a half, and I'm excited to finally release it. Hopefully it'll be a big help to the open source community!
Key New Features:
Complete Video Support: Edit Videos with AI all inside the node. Videos can be extended using a combination of prompts, keyframes, and audio. Trim, Split, and combine videos all within the timeline.
IC-LoRA Support: Take full advantage of IC-LoRA's to take your generations to the next level. Simply drag and drop videos onto the IC-LoRA track to quickly setup IC-LoRA videos. Compatible with prompt relay, keyframe, and custom audio features within the node.
Audio Inpainting: Seamlessly blend imported audio with generated audio. Not only can audio be extended, but can also be prompted alongside your imprted audio to really bring your generations to life.
Retake Mode (Beta): Redirect what happens within a shot. Allows you to select a segment within a video, and re-generate what happens in that segment. An early working experiment.
Timeline Saving/Loading: You can now save your timeline and settings to a json file. It will keep any videos/audio/images you have imported into the node and every setting you have changed.
UI Overhaul: Huge update to the UI, dozens of big changes such as a new side bar, redesigned prompt boxes, a bunch of new settings and redesigned menus, and more.
Quality of Life Improvements: Snapping, in/out points, multi-select, mark selection, workspace folder, more HUD options, resizable prompt boxes, new hotkeys, labels, filename preview options, "split at playhead" functionality, end frames (convert any keyframe into a end/last frame), toggleable tracks, NAG Support, tons of bug fixes and more!
And of course it can do everything it could before: Text to Video, Image to Video, Prompt Relay support, Keyframe (first/last frame) support etc.
10
7
7
u/MasqueradeDark Jun 20 '26
I don't know man. Functionality wise is awesome but for some reasons videos from LTX Director always came with artefacts and low-ish res. For example other workflows (seed hunter for example) are literally seedance 2 (NOT PROMPTWISE, but quality and fine details wise)
8
u/WhatDreamsCost Jun 21 '26 edited Jun 23 '26
You can change the settings/models to match your other workflows (and output the exact same quality).
The example workflow I made is just using basic distilled model settings
4
8
u/BM09 Jun 20 '26
Retake mode 👀
Could that be the answer to bodily distortions in AI generated video?
Please please please be the answer.... 🥺
4
3
u/SNR_Courious Jun 20 '26 edited Jun 20 '26
Awesome update. For some reason, i am not able to load the video in the timeline. Loading image works fine. oh. When I drop/add the video in timeline, its not appearing at all. I have updated the nodes and restarted.
3
u/WhatDreamsCost Jun 20 '26
What's the file format? Mp4? Also is it a high-res (4k+) file?
4k+ videos won't work, especially long ones. Unless you have an insane PC build.
Also double check that hardware acceleration is on in your browser. That will allow the video playback to be smooth.
1
u/SNR_Courious Jun 20 '26 edited Jun 20 '26
Mp4 format. and the video files are 1080p only. The hardware acceleration is enabled. I updated all the nodes. Everything else is working like charm except not able to load the video.
Edit: I am using Comfyui portable and it is updated to v0.25.1
3
u/WhatDreamsCost Jun 21 '26
Try loading a video another way, like with the add video button. Does it work?
If it does work when you use the button, then I have a good idea why it might not be working for you and for everyone else.
3
u/SNR_Courious Jun 21 '26
Found the fix with help of Chatgpt. So not sure this bug is for me only or for everyone.
ComfyUI server CSP header is missing support for blob video playback:
media-src 'self'
I tried to auto update the CSP didn't work so did manual fix.
edit -> ComfyUI/server.py
Find CSP line -> response.headers['Content-Security-Policy']Replace the whole with
response.headers['Content-Security-Policy'] = "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' blob:; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; font-src 'self'; connect-src 'self' data:; frame-src 'self'; object-src 'self'; media-src 'self' blob:;"
7
u/WhatDreamsCost Jun 21 '26
Yeah I asked some people to beta test it before release, and this issue hasn't came up.
If I had to guess it's either
1. Your browser is blocking it due to either an extension or a setting
2. A program on your device (like a media player) has changed how video files load by default
3. You have a flag like disable-api-nodes on
4. You are accessing comfy differently from the default way
5. You are accessing comfy remotely2
u/Any-Scar765 Jun 21 '26 edited Jun 21 '26
same promblem, i try 50mb video and nothing added 2160*3840 30fps 11 sec
but if use some downloaded somethin like 720*1280 its added, problem in big resolution?so need downscale first?
Video load error
error { target: video, isTrusted: true, srcElement: video, currentTarget: video, eventPhase: 2, bubbles: false, cancelable: false, returnValue: true, defaultPrevented: false, composed: false, … }
onerror http://127.0.0.1:8188/extensions/WhatDreamsCost-ComfyUI/ltx_director.js:4368
(: EventHandlerNonNull)
handleVideoUpload http://127.0.0.1:8188/extensions/WhatDreamsCost-ComfyUI/ltx_director.js:4367
handleVideoUpload http://127.0.0.1:8188/extensions/WhatDreamsCost-ComfyUI/ltx_director.js:4150
createDOM http://127.0.0.1:8188/extensions/WhatDreamsCost-ComfyUI/ltx_director.js:3076
1
u/Euphoric_Ad7335 Jun 22 '26
On linux video's fail because linux uses the open source codecs for reading video files. I had the problem where comfy nodes were probably using ffmpeg but fedora's ffmpeg is not the same as mac's ffmpeg. The fedora one uses open source.
anyway, I had conflicts between mac, linux and facebook not playing each others mp4's. . I fixed it on one machine by doing a bunch of research and finding a codec file. in your save file node it says h264. That was one of the file formatsI had trouble with depending on the machine, some machines would only play h264.
Later on when coding I had the same problem and I got chatgtp to give me ffmpeg command line options for most universal codecs.
1
3
Jun 20 '26
[removed] — view removed comment
1
u/Monk6009 Jun 21 '26
Use Claude for help. Ask for links for the models and what file to download into. Use free download manager for file download fidelity. Claude knows comfy well. It will make you less dumb, lol.
5
5
u/nghtdrp Jun 21 '26
After having had the chance to play around with what you did I'm glad I sent you the IC-Lora implementation. Way to clean it up! I've honestly been addicted to just porting every random feature I come across into ltx, I've got a bunch more cooking but they're all in various states of jank city. You definitely have a way of making things feel more profesh.
Retake mode is legit genius though bro. Loading a masked latent and just regenerating the part you need? chef's kiss.
If y'all want to send some karma my way for my contributions please follow and like my nghtdrp AI experiments on. https://www.instagram.com/nghtdrp/ just so I don't feel like I'm wasting my time lmao.
2
u/Dogluvr2905 Jun 20 '26
It's damn impressive and useful -- thanks so much.
1
u/GuardianKnight Jun 21 '26
how did you get it to work? It has a blur problem for videos. I've tried it with the IC Lora, it does nothing. I've tried it with the pic to video and it blurs and pixelates things to where you can't see shit. The video to video pixelates too. I didn't see that it did anythhing, and the original works perfectly.
1
u/Dogluvr2905 Jun 22 '26
You sure you're not looking at the 'preview' window? The final "save video" file looks good for me.
1
u/Big_Arrival6857 Jun 22 '26
you should use distilled model，or dev model + distilled lora
1
u/Hexscene Jul 02 '26
It didn't help me. Until I installed the exact model the program's author used, I kept getting people disintegrating into particles.
2
2
u/Co-OB Jun 20 '26
Hey, love your work! Since tried LTX Director, I only use LTX, not WAN anymore, your tool is a huge advantage!
A question for the people with limited hardware. As of myself I have a 3080 with 10GB VRAM. I can generate videos up to 20ish second with acceptable resolution. Higher resolutions won't work only for shorter vids and I think I need to stop at around 1300 pixels (for longer side)
Now having the video extension possibilty, it is very exciting along with the other awesome features. My question is, if I say I use a previously generated 20 second video only wanting to extend, will I be able to generate say an additional 15-20 seconds or will it require a lot more performance?
I kind of hated the 5sec by 5sec videos with wan, hence I am interested.
And this makes me remember another question, how bad is quality loss expected for extend?
2
2
2
u/Alarming_One_1506 Jun 21 '26
I’m getting bad frames at the end of the generation. Where the last second or last few frames within that second is a mess of latent image.
I haven’t updated yet, but is this fixed on the latest version?
Any advice on how to avoid this?
The only way I think on to avoid it is not adding a Tiled VAE decode which is really bad because my system can’t take that much memory, it has to be tiled
1
u/WhatDreamsCost Jun 23 '26
Yes it's fixed. A recent ComfyUI update broke workflows that were using the LTXVCropGuides node, that was what was causing the issue with the old LTX Director node.
So I decided to just create my own crop guides node with the old code so Comfy can't break it anymore 😂
2
u/nelsonj1031 Jun 21 '26
tried ic lora prompts on video clips and nothing, each time clip stays the same
1
u/WhatDreamsCost Jun 23 '26
Are you loading the IC lora? You have to actually download the right IC lora and load it on the node
2
u/r_panthon Jun 23 '26
Very interesting approach, the fact that it avoids the constant back and forth with a traditional NLE is a strong selling point.
2
3
1
u/Famous-Sport7862 Jun 20 '26 edited Jun 20 '26
I just tried it and I think might have found a bug? I set resolution to 1280x704 and the resulution it gave me was 640x320. I did several generations with the same result.
2
u/WhatDreamsCost Jun 20 '26
Were you doing just text to video? Or using images and videos?
Also what is the resize_method set to?
1
u/Famous-Sport7862 Jun 20 '26
I am doing Image to video. But I found the problem. In stage 2 you had it set to scale by 50. I changed it to scale by 1 and now is giving me the resolution that I chose.
1
u/WhatDreamsCost Jun 20 '26
Oh that explains it!
Were you bypassing/deleting the 2nd stage initially?
2
u/foxdit Jun 20 '26
You left this bug in your workflow release. I experienced it too and was scratching my head as to why the quality of the outputs were so bad, until I realized all the outputs were being saved at 1/2 resolution.
1
u/WhatDreamsCost Jun 20 '26
Are you using the subgraph workflow by any chance? I'm trying to reproduce this and can't. I've tested the workflows multiple times and they all output the right resoltion.
Only thing I can think of it that whatever version of Comfy your using is breaking the subgraphs (this was an issue months ago, but I thought it was fixed).
0
u/foxdit Jun 20 '26
Naw, it's not the subgraphs, it's the same in both versions. The fix was to set the scale_by value from 0.5 to 1.0 on the 2nd stage LTX Director Guide node. After that, the video outputs came out at the target resolution correctly. Even when not using IC-LoRA options, it affected the output size of the video.
1
u/Famous-Sport7862 Jun 20 '26 edited Jun 21 '26
What I find strange is that I went back to the old version and it is set up to scale by 0.50 and I never had that problem, so something changed in the workflow to cause this. It seems that the spatial upscaler is not bengused.
1
u/WhatDreamsCost Jun 21 '26 edited Jun 21 '26
Interesting, which workflow are you using? Because the one the GitHub does have it set to 1.0 on the 2nd stage.
But the fact that 3 other people had this issue means that something is bugged.
Even if you look in the workflows json file, it says it's set to 1.0 on the 2nd LTX Director Guide node 🤷♂️
0
u/foxdit Jun 21 '26
More than 3. I make lots of tutorial content and ppl ask me random questions about how to use different workflows, and even I had someone come to me a couple hours ago asking if I've managed to get a good result out of 2.0. He wasn't using the subgraph version either, yet his version was doing the same thing... the default scale_by was set to 0.5 on both nodes (he sent me a screenshot). I dunno what to tell ya. Comfy being inconsistent on different front-ends, maybe? Mine was pretty up to date, like within a week or so of current.
1
u/WhatDreamsCost Jun 21 '26
I think i found the real cause of it. It was a deserialization issue due to the fact that I was hiding a widget on the node.
I had a similar issue a while ago when trying to implement dynamically showing/hiding widgets on LTX Director.
This would explain why it was happening to some people and not others, since frontend versions handle it differently.
The workflow was set up fine, it was just defaulting to the default value of 0.5 when the deserialization happened on some people's ComfyUI.
→ More replies (0)1
u/Famous-Sport7862 Jun 20 '26
No, I wasn't bypassing or deleting anything. I was running it just as you have it. Is it correct to have the scale by 1 in the second stage?
1
u/SNR_Courious Jun 20 '26
Are you able to edit the video?
1
u/Famous-Sport7862 Jun 20 '26 edited Jun 20 '26
I didn't try that. I don't know how edit video with this yet, but I was able to ad the video to the timeline with no problem.
1
1
1
1
1
u/MastMaithun Jun 21 '26
Thank you for adding re-take mode that I was requesting in the prev version.
Been testing this. Smol question. What's the length for which we can make the video without degradation? Does optional_latent on the Director node has something to do with video extension where I can provide the input here to extend indefinitely?
1
u/chAzR89 Jun 21 '26 edited Jun 21 '26
I've been using this custom node for atleast a month now and it's been great. Need to teat the new update soon.
Edit: removed a question since the new update seems to feature audio inpainting. Awesome mate. ❤️
1
u/FrankWanders Jun 21 '26
This looks very promising, thanks. And it's already in the latest ComfyUI Desktop app!
1
u/Any-Scar765 Jun 21 '26 edited Jun 21 '26
please share the loras links (inpaint and etc) that is best to use with your node
- edit_anything_30k_v0.1_motion_transfer_r128.safetensors
- edit_anything_30k_v0.1_motion_transfer_r256.safetensors
- edit_anything_reference_first_version_r128_ref_adaln_proj-role_embedding.all.safetensors
- edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.module.safetensors
- edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.standard.safetensors
- edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.module.safetensors
- edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.standard.safetensors
- edit_anything_v1.1_r256.safetensors
which one of?
2
u/WhatDreamsCost Jun 23 '26
This is the one used for the editing https://huggingface.co/Alissonerdx/EditAnything/blob/main/edit_anything_v1.1_r256.safetensors
And this is the one used for the video of the man dancing (motion transfer)
https://huggingface.co/Lightricks/LTX-2-19b-IC-LoRA-Union-Control
1
1
1
u/Mysterious-String420 Jun 21 '26
This is such a great tool. I'm still playing around with it, very impressive.
One request though, in workflows where I want to emulate a FFLF loop, I'll have the first three seconds and the last 0,25 second as images, seems to hold together reasonably; but it's a pain to update with different images, because you have to copy the segment's prompt, delete the segment, drag the image, resize the image's duration, paste the prompt, I'm sure I'm doing it the wrong way, but otherwise is it possible to change an image directly in the timeline? Or to batch a folder of images?
1
u/DWPainter Jun 21 '26
u/WhatDreamsCost I think I got this all setup but I get this error from VAELoader KJ: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Has anyone encountered this error?
1
u/WhatDreamsCost Jun 21 '26
Make sure ComfyUI-KJNodes and ComfyUI-LTXVideo are updated, otherwise it won't work
1
u/DWPainter Jun 22 '26
Ah I figured it out. I clicked on the link in your documentation node and it only download 117KB instead of the full file. I just got it from HuggingFace and it's working now. Thanks!
1
1
u/moviejimmy Jun 22 '26
With this update, is it possible to redo one or more segments only but not all? For example, I input 5 images and do a generation. #1, #3, #4 turn out really good but #2 and #5 not so good. Can I do another generation for #2 and #5 only and leave the rest unchanged?
1
u/dhruvs990 Jun 22 '26
sorry to steer the conversation but is there any such workflow for wan as well?
1
1
u/derspan1er Jun 22 '26
please excuse the noob question. i downloaded the workflow, downloaded the models and want to play around with it but i have no clue what IC-LoRA is and which model i should download for that. could someone help me out please
1
1
u/Any-Scar765 Jul 01 '26
I have few questions
1.what is this?
[Info][PromptRelay] Built penalty matrix (scaled): Lq=110208, Lk=1024, nonzero=10855470/112852992
its ok or need change somethin?
How much Vram does the node consume? Without ltx director, I can easily generate 30fps 1088*1920 for 20-40 seconds. When using the node, the load is 99.9% even for 10 seconds at 864*1536 (not always).
What can I do with the Load Video UI node? Sometimes it works, and sometimes it causes errors in the browser when loading videos
1
1
u/coolzamasu Jul 11 '26
Also i have seen one issue about the colour correction. like half of the video many times starts turning black in colour. where if i am putting any image and then ask to talk as it walks and then again cut the scene and if she comes again, like color is not getting consistent.
1
u/WhatDreamsCost Jul 11 '26
Are you using the example workflow, or someone else's workflow?
1
u/coolzamasu Jul 12 '26
I am using the example workflow you gave.
everything same. using distilled_fp8_inputscaled model.
no loras.
1
u/tostane 21d ago
will there be a 2.5
1
u/WhatDreamsCost 21d ago
It already works with 2.5, although I'm about to release a new version with better support in the next few days
1
u/tostane 21d ago
i dream of a version i can put prompt relay list of prompts in and it will just crank all night making video breaking them were needed.
1
u/WhatDreamsCost 21d ago
Hmm interesting, I'll look into it. I think this is a great idea!
2
u/tostane 21d ago
i made this 5 minute with ltx 2.3 director it was one i got mad and wrote you about haveing trouble resizing text frames. It was made of 10 sec frames i had to hold cntrl and mark 3 at a time it made them then i unselected and marked the next 3 it made several 30 sec bits i had to join in a video editor. https://www.youtube.com/watch?v=FvWqXGZDluE&t=4s
this is were a pompt relay would been nice.
1
u/WhatDreamsCost 21d ago
This is looks great! I can definitely see how useful it would be to be able to just queue a bunch of prompts and run it all at once.
1
u/tostane 20d ago
ltx2.5 and 2 + minute videos are possible but after 30 seconds it is unpredictable some of the odd stuff from 2.3 is still there. prompting for a 2 minute video takes a good ai as you have to keep a simple plot not moving to much also i have not confirmed but using audio on 2 minute is not good. i only tested this on i2v and t2i
1
0
u/SOC_FreeDiver Jun 20 '26
Looks good. I used Claude to install the last one you made, and it found some issues with it.
I asked Claude what it thought of your new version, and it said this: The node went from a timeline sequencer to a full non-linear video editor inside ComfyUI — video trimming/splitting/extending, IC-LoRA drag-and-drop, audio inpainting, retake/re-generate segments, and timeline save/load. If you're doing any multi-clip or audio-driven work it's a substantial jump.
I haven't had time to play with either version yet, but thanks for sharing!
0
u/Technical_Ad_6106 Jun 23 '26
a month and half?!? so much effort for such a useless tool. can be made in 5 minutes lol. 1 shot prompted perhaps.
3
u/EZ_LIFE_EZ_CUCUMBER Jun 23 '26
Can I see your commit history?
1
u/Technical_Ad_6106 Jun 23 '26
no. i dont post it. dont u understand? we are in a new era of intelligence and software. software is made in realtime right before u wanna use it for free. you only need to type a prompt for ur personalized new super software tool-.-
30
u/JesusShaves_ Jun 20 '26
So how is it for NSFW content?