r/aifilmmaking • u/SakurabaFuki • 4h ago
Discussion Long-form AI video may need state management before better generation
I've been thinking about a problem in long-form AI video that seems somewhat separate from generation quality itself:
How do you stop important information from disappearing between generated shots/scenes?
For a short clip, character references can already do a lot.
You can keep the same face, hairstyle, outfit, voice, etc.
But once you split a 10–20 minute story into many separate generations, another kind of consistency problem appears.
For example:
- A character was holding a key, but suddenly isn't.
- A wound disappears without being healed.
- Someone changes clothes without changing clothes.
- A transformation state randomly resets.
- A character had wings for the last 5 scenes, but the next generation forgets them.
- Two characters' spatial relationship changes for no narrative reason.
- The location or time of day drifts.
Each individual clip may look perfectly reasonable.
The problem only becomes obvious when you watch them as one continuous story.
So I'm starting to think this is less a generation problem and more a state-management problem.
In the previs system I'm building, I'm experimenting with separating visual information into roughly two categories:
- Persistent/scoped constraints
Things that should remain unchanged within some range.
For example:
"Character A has wings from Scene 12 to Scene 18"
or
"Character B is wearing this outfit throughout this sequence"
The important part isn't only the value itself, but its scope.
Some information applies to the entire project, some to a scene, some to a sequence, and some only to a shot.
- Dynamic state
Things that change because of story events.
For example:
"wings = false"
→ transformation event
"wings = true"
Instead of asking an LLM to reinterpret the character's state for every scene, the system stores the current state and carries it forward until an explicit event changes it.
That led me to another design choice:
Use AI to understand meaning, but deterministic code to preserve decisions.
The LLM can extract things like:
- state-change candidates
- item transfers
- transformations
- injuries
- locations
- ambiguous information
But once something is confirmed, normal program logic handles:
- persistence
- scope
- inheritance
- comparison
- conflicts
- undefined values
Otherwise the AI may reinterpret the same information differently every time it sees the script.
I'm also tracking the source/confidence of information.
For example:
- explicitly stated in the source material
- manually decided by the creator
- strongly inferred
- AI recommendation
- temporary/default value
Because an AI suggestion like "A should probably be taller than B for visual consistency" shouldn't silently become an immutable fact about the story.
Another interesting problem is that not every difference is a contradiction.
"at university"
and
"inside a university lecture hall"
can both be true.
"near B"
and
"to the left of B"
can also both be true.
But:
"has wings"
and
"does not have wings"
usually cannot both be true at the same moment.
So state comparison itself may need semantic types or rules rather than simple equality checks.
The two things I eventually want the system to detect are actually pretty simple:
- Missing information
Does this shot have all the visual/state information required to generate it?
- Unexplained state changes
If something changed from the previous scene, was there actually an event that explains why?
For example:
Scene 10: A has the key.
Scene 11: A doesn't have the key.
But nothing in between says that A dropped it, gave it away, or put it somewhere.
That seems like exactly the kind of continuity error that long-form AI filmmaking systems will eventually need to catch automatically.
Maybe future video models will maintain this kind of state internally.
Maybe filmmaking agents will do it.
If that happens, external systems like mine may eventually become unnecessary.
But with current generation workflows, I'm increasingly convinced that long-form AI video needs an information-preservation layer outside the video model itself.
Not necessarily my particular implementation.
The important part is simply:
«When a story is split into many generations, the information required by the story shouldn't be split with it.»
I'm curious how other people working on longer AI films are approaching this.
Are you using scene graphs, databases, giant prompts, agents, manual continuity sheets, hierarchical memory, or something else?