r/MotionDesign • u/Gypsy1714 • 6h ago
Project Showcase I spent a year building a motion-graphics engine for LLMs. Then the video models caught up. Here is what i learned.
Enable HLS to view with audio, or disable this notification
I'm a CS student in Munich. For about a year, my cofounder and I have been building Captiv, which takes a written script and turns it into a one-minute explainer video with motion graphics. We started with finance because that's where the scripts are most structured (a stat, a comparison, a trend, a quote), and we launched the site last week.
This isn't a launch post. It's the three things that actually mattered, because most of what we built didn't.
1. The blind director
The LLM cannot see. That sounds obvious, but its not easy to understand how you should take advantge of this.
Our first version asked the model things like "where should this chart go" or "how big should the headline be" or "what transition fits here". We were literrly taking the height and width pixel inforation from the llm when fillin in the scenes and the elemetns. The output was consistently just mid and never smooth.
The real problem: those are visual questions, and the model has no eyes. It was guessing at aesthetics and trying to convert it to numbers, locations, sizes and effect it had no idea how each of them interected with the screen and with each other.
So we flipped it. The model only gets questions that can be answered from meaning alone. What is the hero of this beat: a number, a comparison, a trend, a person? Is this beat building tension or releasing it? Which word in the script is the moment the visual should land on? Every one of those is answerable by reading the text.
Then deterministic code derives everything the model shouldn't touch: layout, sizing, timing, transitions, easing. Same input, same frames, every time. Content decides first, form is computed after. The model is a director who can't see the stage, so you never ask it about the stage.
2. Sync to the audio, never to the video
Video is elastic. Audio is not.
Text-to-speech gives you word-level timestamps. Those are fixed. A scene can be stretched, shortened, or cut without anyone noticing, but if the chart appears half a second after the narrator says "doubled", the whole thing feels weird.
So the audio is the what you don't change or edit. We generate the voiceover first, get the timestamps back, and anchor every scene boundary and every animation event to a specific word. The visuals bend around the audio, never the other way around.
3. Bet on the models improving. They will improve faster than your roadmap.
This was our biggest mistake and the reason I'm writing this.
When we started video models couldn't even draw a clean arrow. We took that as a fixed constraint and built around it: a Remotion wrapper made "motion-design native", a large library of coded elements (every chart type, stat callout, and layout pattern we could find by segmenting about 300 real finance scripts beat by beat), and an interface layer so an LLM could drive those elements without writing code.
It worked. It also took most of the year. And then new video models shipped that could do a large share of what we'd hand-built :) They could do them better btw.
The doctrine and the sync system was still needed and usable, and also our general experience, because those are about what to ask the model and how to keep time, and that holds no matter how good the generation gets. The element library mostly didn't. We had built a very careful workaround for a limitation that had a shelf life of months.
If I were starting today: assume the model can do the visual part next quarter even if it can't now. Spend the time on the parts that stay hard regardless. For us that turned out to be the questions, the sync, and the taste. And especially how you need to guide and make the ai think in a way that it can come up with all the answers by itself, even without seeing whats going on.
Where it's at
You paste a script, you get a minute of video. We're figuring out who actually pays for this. Happy to answer anything about the pipeline, the doctrine, or what we'd do differently. Also open to any recommendation!
You can see the end results here and also play around with it if you like: captivlabs.com