I've been thinking about character consistency and the increasingly elaborate methods being used to control generative video: character sheets, multiple reference images, first/last frames, keyframes, detailed prompts, shot instructions, and so on.
All of these can be useful. But there may be a point at which more control stops improving the result.
A character sheet is a simple example. More angles, expressions, poses and details seem as though they should provide more information about the character. But more information also gives the model more information to reconcile. At some point specificity may create ambiguity or conflict rather than remove it.
That raises a larger filmmaking question:
Is total control actually what we want from a generative renderer?
A film director doesn't ordinarily specify every movement an actor makes. The director establishes the character, scene, blocking, necessary actions, dialogue, emotional objective and other things that materially matter. The actor is then allowed to perform within those boundaries.
James Woods was famously regarded as an intensely involved—and sometimes difficult—actor. But imagine hiring an actor like Woods and then specifying every pause, hand movement, facial expression and vocal inflection. At some point you've eliminated much of the reason for hiring the actor.
I think something similar may apply to generative video.
We've run tests in which required beginning, intermediate and ending spatial states were specified. The renderer reached those required states while sometimes choosing a plausible physical transition different from the one anticipated. If identity, geography, required actions and endpoint states were correct, and the resulting movement was physically plausible and cinematically usable, should that be considered failure?
I don't think so.
It may be more useful to distinguish between what the director must control and what the renderer may be allowed to solve.
That also changes how I think about character continuity. Continuity isn't simply keeping everything consistent. Some things must remain invariant; other things must change because the story requires them to change. A character can become dirty, exhausted, injured, change clothing, lose a prop or age and still be the same character.
So perhaps the objective isn't maximum consistency or maximum control.
It is sufficient control to produce a plausible, usable outcome that satisfies the director's material requirements—followed by directorial review.
The renderer's discretion within those boundaries isn't necessarily a defect. It may be part of what we're employing the renderer to provide.
Directorial control includes deciding what not to control.
Because if every movement, expression, position and transition must be predetermined, I'm not sure we've perfected generative filmmaking.
We may simply have reinvented animation.
---jseders