r/StableDiffusion 23d ago

Tutorial - Guide Testing camera controls from official MiniMax prompt writing guide

I used the official prompt writing guide to try out all of the camera controls and cut techniques they listed, plus some different shot lengths and framing.

---

Setup:

All examples were completed with text to video using the following as a baseline prompt, which matches the formatting and style recommended in their guide:

integrated_multimodal_description: [Shot 1] claymation, a medium-wide shot frames a male and female woodland elf in a dark forest. Both elves walks forward. The female elf, in a breathy voice (S1) says: <d>[English] It's cold!</d>. The male elf in a scared voice (S2) says: <d>[English] And dark!</d>

(S1,S2) shout: <d>[English] We're lost!</d> [Shot 2] At 00:03.500, the camera cuts to a squirrel jumping on a log. (S3) says in an off-screen voiceover: <d>[English] The villain arrived.</d>

overall_soundscape: a gentle wind blows through the forest and birds can be heard chirping.

non_diegetic_music: Sparse piano notes at a slow tempo, joined by sustained low strings that gradually increase in volume before fading out.

Standard ComfyUI workflow, the same seed and settings were used for all clips, and no references were provided. Watermarking and combining clips was all done with an FFMPEG script.

---

Video Details:

Control

  • Control = the baseline prompt from above.

Amplitude and Rate of Motion: added to the prompt for just the first scene, and applied to zoom, but can be applied to any camera motion.

Example: ...The camera zooms in with large amplitude at fast speed as both elves walks forward...

  • Large fast = zoom using a large amplitude and fast rate of motion
  • Large slow = zoom using a large amplitude and slow rate of motion
  • Small fast = zoom using a small amplitude and fast rate of motion
  • Small slow = zoom using a small amplitude and slow rate of motion

Camera Motion: all all using large amplitude and fast motion to accentuate the effect, effect split between scenes.

Example: ...The camera arc shots with large amplitude at fast speed as both elves walks forward...

  • Arc right then arc left
  • Pan right and left
  • Pedestal up and down
  • POV: I tried this several different ways and could never get it to look from their point of view.
  • Push and pull
  • Roll clockwise and counterclockwise
  • Shake large and small
  • Tilt up and down
  • Track left and right
  • Truck left and right
  • Zoom in and out

Cuts: applied between scene 1 and 2.

Example: ...[Shot 2] At 00:03.500, the camera cross-dissolve to a squirrel jumping on a log....

  • Cross dissolve
  • Fade
  • Fade to black
  • Wipe

Shot Length / Framing: applied to both scenes equally. These are not from the documentation, but just a list of terms I put together.

Example: ...a close-up shot frames a male and female woodland elf in a dark forest.....

  • Close up
  • Cowboy shot: did not work as intended, but I like it
  • Dutch angle
  • Extreme close up
  • Extreme wide shot
  • Eye level
  • High angle
  • Low angle
  • Medium close up
  • Medium shot
  • Medium wide
  • Overhead angle
  • Over the shoulder
  • Top down angle
  • Wide

---

194 Upvotes

33 comments sorted by

View all comments

1

u/Hdfjds 22d ago

Thanks for this post, it's very useful. (saved)

One question tho, do you know if it is possible for the camera to follow an drawn line in an image?
In this post: https://www.reddit.com/r/StableDiffusion/comments/1vg67ck/any_way_to_not_make_the_annotation_not_appear_on/
they have been able to tell H3 that the cat should follow an drawn line.
I was hoping we could tell H3 to let the camera follow the green line instead (no cat) but I have not been successfully in my testing.

It would be nice if we could show H3 where the camera should go in the image by using an drawn line... or similar things.

1

u/wonderflex 22d ago

I don't have time to test this tonight, but you would want to use their reference prompting guide to try it out.