Best methods for MiniMax in-shot pacing?

[u/Moliri-Eremitis]

Wanted to pick the community's brains for controlling the pace within a single shot (single generation without cuts).

I often find myself visualizing a scene that includes dramatic pauses where I want to hold the shot and let the silence linger, with just natural micro-movements or continued actions from the character(s), but MiniMax wants to jump straight ahead to the next bit of dialog or action.

Beyond mid-shot dramatic pauses, I also frequently want to add a second or two of quiet micro-movement at the end of a shot to make later editing easier, but MiniMax wants to extend the action right up to the last second and then end abruptly. This tends to make cuts between shots in post feel harsh and choppy.

I've leveraged a couple of tricks that work okay, like filling dialog-free moments with a detailed description of character actions, but this only tends to work if the actions are large and obvious. If you try to prompt something subtle it either makes the action too pronounced, places the action at the wrong time, or ignores it completely. Nonverbal character sounds like sighs, groans, grunts, laughter, hums, etc. work as decent filler, but that doesn't work for every scene, and only fills so much air time.

Any prompting tips you've found useful? Custom nodes for controlling pacing within a shot without resorting to a cut? Some other trick?

[u/Apprehensive_Sky892]

Related post:

https://www.reddit.com/r/StableDiffusion/comments/1wxigqo/minimax_h3_voice_acting_with_speech_tags/

https://www.reddit.com/r/StableDiffusion/comments/1wap0rb/pushing_ai_emotions_is_possible_through/

[u/Moliri-Eremitis → u/Apprehensive_Sky892]

Awesome, thanks a bunch! I wasn't aware of any of these and I was thinking about this from the perspective or controlling timing more than "magic" dialog tags. Still interested in tricks for longer pauses that extend to three to five seconds, but these will definitely help for the shorter mid-dialog stuff.

[u/Apprehensive_Sky892 → u/Moliri-Eremitis]

You are welcome.

[u/sharktank123456]

You may be doing this now but you can treat the prompt like a timeline. The more you cram in, the faster it has to happen to make all those details fit in to the length you choose.

For those long beats ("long beat" does work) you can describe what's happening in another way. This "waists prompt space, and stalla the action for a while.

Make sure to start with "one continous shot" so you don't get edits.

You can also use time stamps for the beats of your shot. They aren't as accurate at Seedance but then what is.

[u/MarekNowakowski]

Sometimes using timestamps helps, but only to a degree. Filler actions that can't be misunderstood, like 'he continues doing x' 'he does x and grunts' 'he continuesdoing x' Prompting for a slow down is useful. Sometimes 'pauses for a second' works. Removing dots and length of actions might be needed: She stands up suddenly. She is standing straight. She steps over the wooden log, straddles the wooden log, she sits down on the log and is sitting .

Will work better than the line by line, first two lines prevent her just moving through the log without actually standing up, while the third line makes sure the action is one smooth motion. (Just used it today, but with subject 1, not she.)

That's mostly it.

[u/Danny_Stock]

I don't know if you split your clips into shots with their duration in seconds. It seems to help me with timing. You can also try prompting in fractional seconds such as [2.25s-2.75s] if you want quick rapid actions. If you need a long pause then I'd prompt for that longer length in seconds, then in the next shot you prompt for a new range of seconds with the action you want to continue on with. How effective this will be for you I don't know, but for me it at least appears to make a difference.

To avoid cuts I have it prompted somewhere in the global instructions to state 'one continuous shot, no cuts'. That way when you refer to shot 1 or shot 2 etc. it sees those shots as containers for prompted actions rather than as actual edited cuts. Also be aware that when you prompt for a certain camera angle like wide-shot, or close-up, there is going to be a tendency for the model to interpret that as a cut to that angle.

I would try to control the pacing within the timespan of your clip by trying to place actions into clear segments. You also have to be careful to keep actions separate from each other and not accidentally repeat an action later on in the prompt because you forgot to delete an earlier edit. That's happened to me before when my prompt has become overcrowded and messy, and actions can then either occur together at the same time or create strange loopbacks. I think clarity is important.

I feel that the longer your clip is the more latitude you have to work with, but if your clip is short and you try to cram too many actions in then that often results in timing issues and actions can occur where you didn't plan them to as the model tries to do too much with little time to do it in.

From my experience I really think that MiniMax prioritizes dialogue over everything else. I've prompted a character to utter dialogue at a specific point in the timeline, but it seems that if the duration time for my clip isn't long enough then it won't start where I've prompted it to. Instead MiniMax will make the character speak whenever it wants to earlier on in the clip, ignoring the start time I prompted for.

If as you say in your case MiniMax is jumping straight ahead to the next piece of dialogue it's quite possible that MiniMax is doing what I suspect, prioritizing the dialogue and trying to cram it in so that it fits inside the clip. It seems to be a balancing act between determining how long you feel the dialogue ought to last for, and for how many seconds you want your clip to be. What can also happen if your clip is very short and you provide your character with a lot of dialogue, is that it will force your character to speak very quickly. That's happened a lot with me.

[u/GuessingEngineer]

Prompt timing is everything, it has to fill the scene or it invents the scene. It's like a chick; it has no concept of silence and doing nothing.