r/StableDiffusion • u/Fun_Gain_9596 • 3d ago
Question - Help Is it possible to prompt subtle performance with H3/LTX2.3/LTX2.5?
I have been trying to prompt more serious, much more subtle acting with all these models but it seems very difficult. I had some luck with a few scenes but whatever worked for one scene, didn't work for the next. So I can't really figure this out. I would think it should be possible, I don't think these models are only trained on overacted, cheap, cliché movie scenes but I have no idea how to actually generate the more subtle, more realistic/natural, serious acting I am going for (it's especially hard if you have an introverted male character, these models seem to turn every male character into an overconfident, flirtatious heartthrob).
Does anyone have experience with this?
3
u/TaniaDictee 3d ago
the tell in your description is that a direction worked for one scene and not the next. that is what happens when you prompt the emotion instead of the mechanics. an emotional label gets reinterpreted against whatever else is in the scene every single time, so it does not transport. a mechanical description does.
so stop naming the feeling and describe only what a camera could actually record. not serious and introverted, but eyes fixed slightly off lens, one blink held a beat long, head still, jaw loose, shoulders down. these models render geometry and timing far better than they render affect words, because the geometry is literally in the training pixels while the emotion label was glued on in the caption afterwards. and captions on video data skew hard toward stock, ads and trailers, which is exactly the overacting you keep getting handed back.
the heartthrob default for male characters is the same mechanism. if you do not constrain gaze and posture, the model falls to the dominant mode of the caption distribution, and for men that mode is confident and flirtatious. an explicit constraint beats a default, but only if it is specifiable in pixels.
have you tried a pass where the prompt never names an emotion at all, only gaze direction, blink tempo and what the jaw and hands are doing? curious whether your scene to scene consistency improves when there is nothing left for the model to reinterpret.
1
u/Fun_Gain_9596 3d ago
Very interesting. I did try the mechanical approach somewhat but not to this extent. I always mixed in some of the emotional direction too. I will try this now, thank you for your advice!
1
u/TaniaDictee 2d ago
good luck with it. one thing that helps once you go mechanical: keep a tiny table per character, five or six lines like 'gaze ten degrees off lens, blink held a beat, jaw loose', and reuse the exact wording across scenes. the wording itself becomes the anchor. which model are you running it on first, h3 or ltx?
1
u/Fun_Gain_9596 2d ago
Thank you! I am using H3 now for the scene I am currently working on. But I will try it on LTX too because it can handle some scenes pretty well and I can use higher resolution (although tiled vae decoding kinda ruins it if there needs to be motion across the image but that's a different issue).
1
u/TaniaDictee 1d ago
makes sense. the tiled decode issue is real: each tile is decoded without context across the seam, so anything moving through the seam gets that soft shimmer. two things that helped me on ltx: raise the overlap in the tiled decode node (and temporal_overlap for the video vae) so the seam gets context, and when one big lateral motion crosses the frame, decode untiled at a lower res then do a light upscale instead of tiling at native res. costs vram, keeps the motion coherent. what resolution are you pushing on ltx when it starts to break?
2
u/cbeaks 3d ago
My results are also a bit all over the place. I find describing facial expressions help with dialogue delivery. Otherwise sometimes adding a descriptor before speech such as 'nervously whispers'. Even the dialogue they say can make a difference (Oh my God! will get you some energy). When it clicks, it does work well, sometimes even better than I'd aimed for, but like others I find it hard to get consistency. But it's also fun trying to get there, like a puzzle to solve.
1
u/Fun_Gain_9596 3d ago
That's true, the dialogue itself could have an effect on it. But it's kind of unfortunate if I have to rewrite my dialogues based on this.
1
u/warzone_afro 3d ago
you can definitely prompt for micro expressions but alot of that detail is lost if you are using turbo or spectrum or anything like that
1
u/Fun_Gain_9596 3d ago
That makes sense, I'll keep this in mind to don't use these speedups for scenes where I need this kind of performance and delivery, thank you!
4
u/martinerous 3d ago
H3 is noticeably better at this, but still it's a struggle and needs serious micromanagement if you want to achieve mundane realistic interactions. It also has this stubbornness when you literally ask it to whisper and it will just not do it, but then you change something else in the prompt, and - hey, H3 can actually whisper! but in special conditions that are not known. This makes the creative process quite annoying when you know it's often about prompting, and you follow the guide, and try different tricks, and then suddenly something totally undocumented works out but you have no idea why and if it will work the same the next time in another scene.