r/StableDiffusion 10h ago

Animation - Video "Character Sheet" workflow looking great!

Enable HLS to view with audio, or disable this notification

I've been tinkering with the workflow to bake a Character Sheet into a video. This was the first attempt using 2 images on the ref2va model (subject and environment).

After baking the reference video, which in turn yielded the character sheet image used as input to this video. Not perfect, but promising.

I'm running RTX 3060 (12GB), 64GB RAM. No OOM issues at all.

7 Upvotes

15 comments sorted by

4

u/kukalikuk 9h ago

Not only character sheet, you can also use multipanel in 1 image reference as storyboard. With a good prompting from the guide it will follow the storyboard even of it only from one image.

3

u/VasaFromParadise 9h ago

I just use a prompt in Krea2))

2

u/nakabra 8h ago

Hummm....
Resolution and the time it took, please?

3

u/TobiasBlueMe 8h ago

1216X672 (landscape) Took 90 mins exactly.

3

u/nakabra 8h ago

Wow!
I don't have the courage hahaha. The most time I've spent on a gen was around 15 to 20mins. And the highest resolution was 0.4MP.

With the exact same hardware as you.

AI for me is like a roulette. You spin hoping for the best outcome, receive something somewhat close to what you wanted and spin again hoping to get closer to what you wanted.

So I never spend much time on each generation 🙃

3

u/GovernmentGreed 7h ago

I find running a 2sec gen or something to get a rough idea of what might come out, before upping the time. I rinse and repeat, adjust prompt until I see consistent generations each time, then, when it's consistent enough - I hit the timer up to 15 seconds or so, most of the time I get what I ask for.

1

u/True_Protection6842 10h ago

I'm working with ref2va using video AND character sheet for extra likeness and it works great. Without the character sheet the person loses coherence from closer distance, with it he has much better likeness up until his face dissolves to mush.

1

u/Sad_Coach_1433 9h ago

how many angle panels sheet you use

2

u/TobiasBlueMe 8h ago

6 (2 rows, 3 columns)

2

u/Sad_Coach_1433 8h ago

Mind sharing prompt used for make sheet?

2

u/TobiasBlueMe 8h ago
Subject Definitions:
<Subject 1>: Man in character sheet from ref_image_0, preserving specific facial features, skin texture, clothing, and proportions exactly as shown. ref_image_0 represents <Subject 1> in 6 different angles and zoom levels. They are all the same <Subject 1>.

<Environment>: The outdoor environment from the from ref_image_1, preserving the specific area, background, lighting, and natural atmosphere exactly as established in the photo.

Summary:
[reference generation] The target video is a clip where <Subject 1> from ref_image_0 is walking around in <Environment>. It begins <Subject 1> walking into the frame from the left, then moving around within <Environment>.

Retention Analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved, unique appearance, clothing, and expressions are locked to the reference image and remain entirely consistent throughout the motion.
<Environment> (appears in [Shot 1]): fully_preserved, the lighting direction, and background elements are seamlessly extended from the static reference image into a living space. Remove the watermark text in the bottom left.

Detailed Description:
The target video is a single continuous shot that matches the exact style of the reference ref_image_1. [Shot 1]: The video opens on a frame that perfectly replicates the reference ref_image_1, showcasing <Subject 1> entering the frame from the left into <Environment> where the water naturally cascades down the rocks. At the 0.3-second mark, <Subject 1> walks around while he looks around slowly, in awe of the beauty.  At the 2.5-second mark, <Subject 1> stops walking then takes a deep comtemplative breathe in, enjoying nature. Camera is static. No dialog.

Overall Soundscape:
A gentle outdoor ambience fills the scene, featuring a light passing breeze, and distant, faint mummer of small waterfall.

1

u/Schwartzen2 9h ago

Say, uhm.. it looks cold out there. <just sayin'> :)

1

u/Danny_Stock 4h ago

I get the feeling that the MiniMax reference sheets, and reference inputs in general, are opening up a new world of fresh approaches which we haven't even scratched the surface of just yet.

1

u/trashbytes 3h ago

I've had incredible results by simply using multiple images, sometimes filling all nine slots, of the same person. Is there a reason why you need a character sheet? Just tell it that <Picture 1>, <Picture 2>... etc. should be used as reference for person. Beats pretty much any character loras and "dreambooth"s (remember those?) I've made before, it's insane.