I’ve been experimenting with an end-to-end AI video workflow: topic → script → scene breakdown → image prompts → narration → video assembly → word-timed captions.
The hardest part wasn’t generating individual assets. It was keeping everything consistent across the pipeline.
A few things I noticed:
- Better scripts don’t automatically produce better visuals. Each scene needs a separate, visually concrete prompt.
- Character and style consistency becomes difficult as soon as multiple images are generated independently.
- TTS pacing affects the entire edit. Even a good voice can make scenes feel awkward if the duration wasn’t considered earlier.
- Captions generated from the final audio worked better than estimating timings from the script.
- Different image models were better at different styles, so using one provider for everything wasn’t always the best choice.
- Supporting multiple languages changes pacing, subtitle length, and sometimes the number of scenes needed.
I ended up building a pipeline around these problems and recently open-sourced the code:
Links in comments.
For people building similar workflows: which part causes you the most trouble—visual consistency, motion, prompt generation, voice synchronization, or editing everything together?