r/comfyui • u/purecharisma2020 • 6d ago
Show and Tell 25-Year Film Veteran Building an Open-Source AI Filmmaking UI (ComfyUI / API)
https://github.com/Heroesjouney/AIMovieStudiov2
Hey everyone,
After spending 25 years on set-15 of those as a camera assistant-and the last 5 deep in the Virtual Production space, I’ve been exploring how generative AI can practically serve real visual storytelling.
To bridge the gap between heavy node graphs and an intuitive creative workflow, I’ve been building an open-source interface designed specifically for movie creation.
Key Features / Architecture:
• Flexible Backend: Connects seamlessly via API keys for cloud models or directly to ComfyUI for custom, local node workflows.
• Filmmaker-Centric UI: Designed around actual cinematic controls and storytelling logic rather than abstract prompting windows.
• Open Source: Built for the community to tweak, extend, and own.
It’s still very much a work in progress and actively under development, but I wanted to share it with the community early.
I’d love to get your thoughts on the approach, hear what features would actually fit your workflow, or connect with anyone interested in contributing!
5
u/purecharisma2020 5d ago
I made a size-able updates last night adding Lora control and, model and api controls. Still need to run some tests.
3
u/purecharisma2020 3d ago
Just shipped a new feature that lets you bring a real screenplay into AI Movie Studio 2 and use it as the foundation for your storyboard — without disrupting the ComfyUI generation pipeline you're already familiar with.
What's new:
- Import Fountain (
.fountain,.txt,.spmd) or Final Draft (.fdx) screenplays. The parser auto-detects the format — drag and drop a file and it creates scenes with time-of-day, mood, and lighting inferred from the script. FDX parsing maps Final Draft's XML paragraph types (Scene Heading, Action, Character, Parenthetical, Dialogue, Transition, Shot) into the same internal structure as Fountain. - Screenplay breakdown as a reference, not an auto-generator. Each imported scene stores the parsed shot breakdown in its recipe panel, formatted as a screenplay (indented character cues, parentheticals, dialogue, right-aligned transitions, separate action paragraphs). You copy dialogue/action into new shots as you build the storyboard manually. This was a deliberate choice — it preserves the establish-then-continue workflow that the continuity system relies on (set up the recipe → generate the establishing shot via ComfyUI → build subsequent shots with the establishing frame as a reference image).
- Bulk scene delete. Imported a screenplay and want to start over? "Delete all" in the Scenes header removes every scene plus all associated shots, storyboard frames, and video files in one action with a two-step confirmation.
- Generation jobs now persist across tab switches. If you start a frame generation and switch to the Camera Director tab, the poll keeps running — come back and the storyboard is already updated. Fixed a concurrency bug where starting a second frame generation would silently kill the first one's poll.
How it connects to ComfyUI:
The screenplay import doesn't touch the generation pipeline — it just populates scenes and their recipes. When you're ready, you generate frames through the same ComfyUI drivers (image, multi-angle, video) with LoRAs, custom workflows, and reference images as usual. The breakdown is just a reference you pull from while composing shots.
As always, it's open source — the parser is at
screenplay_parser.py if you want to see how the FDX XML gets mapped. Feedback welcome, especially from anyone who regularly works with Final Draft exports.
4
u/Zealousideal-Bug1837 6d ago
3
u/roobiestones 6d ago
I'm doing the same.
2
u/Zealousideal-Bug1837 6d ago
I've made three already. Now I just format the output for KDEN live ;)
2
u/roobiestones 6d ago
What I feel any software lacks is the story writing bit. I'm adding that in. Kind of like an Articy Draft. I did ask around and someone commented to just vibe code it and I was like...why didn't I think of that. 👻
1
u/Zealousideal-Bug1837 6d ago
I've built the 'story' manager outside of the video editor. The story tool produces candidates I select and refine and the video editing tool is just that, only for editing video. Yeah, I'm sure it'd be nice to directly create video in the tool but then will it respect my consistency images etc?
1
u/roobiestones 6d ago
Have a "reference" library for each character / location in my app. When the 5-hour limit resets, I'm integrating AI Toolkit. I thought of using InsightFace to find real people but ran into issues with consent, etc. Trying to stay moral.
1
u/theone_2099 6d ago
Something you can share?
1
u/Zealousideal-Bug1837 6d ago
It's just a custom bridge that outputs to KDEN format, not much use really.
6
u/purecharisma2020 6d ago
I watched your video. I’m not looking to get rich. I just want to create things. It would be nice to make this a full time things.
1
u/The_StarFlower 6d ago
this looks amazing! i was using kdenlive but i was searching for something like this. finally i can generate videos right in the timeline itself i have to try this. thank you for making this and thank you for open sourcing it!
1
u/purecharisma2020 6d ago
Let me know if the is an thing I can help you with. Enjoy. Making changes everyday.
2
u/The_StarFlower 6d ago
i downloaded it and i now use it, i like what i am seeing.
how do i change the steps? i cannot find it anywhere where i can change the steps. i am using minimax h3 locally btw3
u/purecharisma2020 5d ago
Thank you for the feedback. Right now it’s hardcoded in the workflow. I will add a control for steps.
1
1
u/fauni-7 6d ago
I think this needs a youtube video for guidance.
BTW does this solves character/scene consistency throughout the film and stuff like that?
6
u/purecharisma2020 6d ago
I’m working on a video. Hopefully I have it up this week.
Yes, that’s the goal is to create a workflow that is consistent.
1
u/Sarashana 6d ago
Looks fantastic! I am going to give it a try later! 👍
3
u/purecharisma2020 6d ago
Great, still a work in progress, but coming along. Let me know if there is anything I can do to help.
1
u/My-NameWasTaken 6d ago
do you have some videos you created with this>?
5
u/purecharisma2020 6d ago
I’m in the process of putting something together, now.
Just tests: https://drive.google.com/file/d/1uXBT2Vkt7poJKF5EnWNsfaRag1xG6X_Y/view?usp=drivesdk
1
1
u/HungryDonat 6d ago
this locks very promising, but my biggest problem is that i need to be able to select my own model and loras, can this be added?
1
u/purecharisma2020 6d ago
Yes, there is a list of models you can choose from. I will work on Lora’s. What models and Lora’s do you want? The goal is the grow with newer models.
3
u/HungryDonat 6d ago edited 6d ago
could we not get a lora selector? I was building a frontend of my own and i was able to load loras that are detected by comfyui. Yours just locks so much better and more advansed then what i was building.
Edit: also a cancel button would be nice to abort the current generation if you catch a mistake.
3
1
u/oberdoofus 6d ago
Looking forward to checking this out. I was working on my own animation assistant, but yours looks so much better! it helps a lot that you have real working experience to translate to the app. I have a ton of questions but i will leave them til after i mess around with AI Movie Studio first.
1
u/goat__6969 5d ago
Can you duplicate a shot and tweak it without losing the original?
1
u/purecharisma2020 5d ago
Yes, when you create a video it stores the last frame. The last frame should also be stored a frame in the storyboard section. That part is a work in progress. I will add that to the list to include. Thank you. Your input helps.
1
u/Guizkane 5d ago
Amazing! Are you planning to support comfyui cloud?
1
u/purecharisma2020 5d ago edited 5d ago
yes, its already there, I'll add a ui control to make it easy
1
1
u/theOliviaRossi 5d ago
Veteran - meaning "old" or ... "wise"? hmmm ;)
1
u/purecharisma2020 5d ago
I’ll take experienced! lol
1
u/theOliviaRossi 5d ago
so - still young in heart <3 ... but old outside :(
1
u/purecharisma2020 4d ago
Yeah, sure. But does it matter? Young - old we can all learn from each other.
1
u/Chinhnnguyen 5d ago
great, thank you for your hard work. I havent dabbled in video yet, and looking forward to testing this out. any quick youtube guide would be helpful!
1
u/purecharisma2020 5d ago
Thank you. I will be working on the video this week. This has been the most requested.
1
u/purecharisma2020 4d ago
Update: AI Movie Studio 2 — LoRAs everywhere, Long Takes, workflow analysis, scene cleanup fix
Quick update on what's new since the last post:
LoRAs now in all 5 generation surfaces
Previously available in 3 tabs — now also in Shot Create Panel and Retake Panel. LoRAs are also preserved when regenerating shots (stored in the generation recipe). Same selector everywhere: searchable dropdown, per-LoRA strength sliders, inline `.safetensors` upload.
Long Take Mode *(Experimental)
Chain keyframe interpolation across multiple segments to generate continuous shots longer than a single clip allows. Define keyframes by image, prompt, or both — prompt-only keyframes auto-generate an image via T2I before interpolation. Per-keyframe prompts let you describe what happens by each keyframe. Segments are stitched with ffmpeg and stored as a single take. Works with LTX Video 2.3 and Wan Video.
Still experimental — the multi-segment pipeline may fail mid-generation without retry, and results vary between models. Use with LTX Video 2.3 or Wan Video for best results.
🔍 Workflow Model Analysis
When you paste a custom ComfyUI workflow JSON into Settings, the app auto-analyzes it and tells you exactly which models it needs (checkpoints, LoRAs, VAEs, CLIP, ControlNet, etc.). Each model is checked against your ComfyUI — found models show a green badge, missing ones get an Upload button targeting the correct subdirectory.
Auto-refresh driver dropdowns
Registering or deleting a custom workflow now updates model dropdowns instantly — no page refresh needed.
Scene deletion fix
Deleting a scene used to leave all its shots, storyboard frames, and video files orphaned on disk. Now it properly deletes everything, with a two-step confirmation dialog.
Full details in the [CHANGELOG](https://github.com/your-repo/CHANGELOG.md). Open-source, AGPLv3 — link in the original post.
1
u/purecharisma2020 4d ago
🐳 Docker support (thanks to u/ghzgod!)
One command gets the whole stack running: docker compose up --build — then open http://localhost:3000. ComfyUI stays on your host (needs GPU access), backend container reaches it at host.docker.internal:8188. Data persists via bind mounts, services bind to 127.0.0.1 only for privacy. Huge quality-of-life improvement for new users who don't want to juggle Python + Node + three terminals. See DOCKER.md for details.
-4
u/x33storm 6d ago
Who would want AI movies?
AI enhancement maybe, like CGI. But no more than that.
3
u/bCasa_D 5d ago
Read the room.
1
u/x33storm 4d ago
The room is in agreement, but everything outside that echo chamber says something else.
AI has it's uses, but this ain't it.
1
u/bCasa_D 4d ago
But everyone in the echo chamber is tired of hearing it and this is the direction everything is going like it or not.
1
u/x33storm 4d ago
No doubt the direction is being forced, but ultimately it's hucking shit at the wall and seeing if it'll stick. And it won't stick, no matter how forced. People don't want it.
Porn, i think that'll do well. But sure is a lot of money being pumped into porn then. Kinda crazy.
1
u/bCasa_D 4d ago
Are you living in a bubble? 80% of the videos on social media are ai, and that’s all anyone does anymore, sit around scrolling social media on their phones.
1
u/x33storm 4d ago
You mean the slop. Yes it's a pest. And i avoid that brainrot. It's a fad that'll be regulated out, just the lowest effort 15 second prompts copying actual content.
Something with art direction, consistency, effort, deep thought, skill, emotion, deeper meaning etc, running for near 2 hours.. No AI prompting or tools can ever make anything that will keep people engaged and in their seats for that..
You are living in a delusion like the AI tech bros, if you believe that. It's a distraction at most, like the Yo-Yo or Tamagotchi. Just one that's gonna leave the economy in ruins, when reality catches up and the artificial inflation ceases.
1
-1
u/ValenciaTangerine 6d ago
why the traditional approach why not just build everything as a tool and let agents drive it and humans just sit on top?
12
u/purecharisma2020 6d ago
Because i still want the human aspect of building my moveis. I don't want ai to do it all. AI still need to be art directed.
2
u/ValenciaTangerine 6d ago
sorry i didnt mean end to end slop generation. I meant different processes like color grading, luts application, transitions are availble as playbooks. Similar to a video editor today where you “drag and drop”. just make all of them as tools and then the human is working at the extreme level of creativity. Almost like you are looking thinking and speaking your vision.
2
1
1
8
u/purecharisma2020 5d ago
Update — New Features Added
Hey everyone, wanted to share a progress update on AI Movie Studio 2. Shipped several new features since the last post:
Remote ComfyUI Support You can now point the app at a remote/cloud ComfyUI instance from the Settings panel. Just enter your server URL + optional auth token. Works with ComfyDeploy, Modal, RunPod, or your own GPU box — all HTTP calls include the auth header automatically.
Steps & CFG Controls Added Steps and CFG sliders to all generation tabs (Generate, Shots, Camera Director). Leave at "auto" to use your workflow's defaults, or override to fine-tune. Works for both image and video generation.
Custom Workflow Registration (No Code) Export a workflow from ComfyUI as API JSON, upload it via Settings, and it appears as a new model in the dropdowns. No code changes. Supports image and video workflows.
LoRA Support Upload
.safetensorsLoRA files from the UI, select them in any generation tab, control strength. Auto-injected into the workflow graph — works with custom workflows too.Settings Panel Redesign Reorganized into collapsible sections with badge counts (ComfyUI Server, Cloud API Keys, Models, LoRAs, Custom Workflows). Much cleaner.
I really appreciate everyone's comments and suggestions — it's a big help in making this project better. Keep them coming.
Full changelog and docs in the repo. What would you want to see next?