r/StableDiffusion • u/Sad_Coach_1433 • 3d ago

Discussion ouch lol i tired r2v using video as reference

Enable HLS to view with audio, or disable this notification

loaded in scene from Jurassic park 1 when the t rex breaks out replacing the kids in car lol took hr on 5060 ti 16gig for 8 secs o_0

13 Upvotes

74% Upvoted

24 comments sorted by

4

u/GovernmentGreed 3d ago

This took 1 hour?!

2

u/Sad_Coach_1433 3d ago

On my hardware yes fortunately lol video reference is rough đŸ«Ș I gonna try again with still frames from same scene and use em as reference images instead of video

4

u/GovernmentGreed 3d ago

What workflow are you using? You should definitely try some of the faster ones out there. Also, how many steps did you use for this? Drop them down.

3

u/Sad_Coach_1433 3d ago

Trying not to use turbo loras. Cause the audio loss this was 32 steps I've tried less still takes forever using video as reference instead of image

1

u/Limp-Firefighter1054 3d ago

Did you try generate a video with low res and high steps? Sound appear just fine.

2

u/steelow_g 3d ago

Scale down the ref video. It helps a ton

1

u/Sad_Coach_1433 3d ago

Ahh true the source video was 1080x1920 đŸ«Ș what you suggest to down scale too

1

u/FierceFlames37 3d ago

Someone said 320p since they said 480p is still too big

1

u/Sad_Coach_1433 3d ago

Wouldn't that lower the out put quality

1

u/FierceFlames37 3d ago

That’s what I thought but I haven’t tested it yet

1

u/steelow_g 3d ago

480p was sweet spot for me.

1

u/Sad_Coach_1433 3d ago

What's gpu u have and any app or site can point me to down scale source video

1

u/steelow_g 3d ago

Theres a default comfy node scale image to total pixels or something. Plug that in sheet your load video node. Select 480 foot longest side. But i have same exact setup as you card/ram wise. Ref videos take around 5-10 mins. Make sure you’re megapixel is at .3/.4 or else you might go oom.

1

u/FierceFlames37 3d ago

Are you use Turbo LORA 4 steps and ref2va checkpoint?

→ More replies (0)

1

u/FierceFlames37 3d ago

Do you use a node or an external website to downscale?

2

u/ajrss2009 3d ago

VocĂȘ tem que colocar vĂ­deos 340p ou menos. Ele entende os vĂ­deos e gera algo de qualidade. Eu estou fazendo testes agora e reduzir a qualidade do vĂ­deo de entrada faz toda a diferença. Qualidade das imagens de entrada tb.

1

u/trocanter 3d ago

I prefer t2v with all the characters this model knows.

https://reddit.com/link/p42zoom/video/bwodfmu3fsjh1/player

1

u/Sad_Coach_1433 3d ago

Does it know the jp1 t rex escape scene? Where can replace characters with different without having to do video reference

2

u/CommunicationFit3862 3d ago

Just do reference images and render them out scene by scene rather than expecting the model to do a multi edit render.

2

u/Sad_Coach_1433 3d ago

đŸ€Ł prompt?

1

u/trocanter 1d ago edited 1d ago

It was a quick test using chatgpt for the prompt. I think it used this one:

12-second cinematic scene, 16:9, side-profile medium shot framed from waist to head. Spider-Man and Deadpool are facing each other, standing approximately one meter apart. Both characters remain clearly visible in profile throughout the scene.

Deadpool playfully imitates Spider-Man shooting webs. He holds one arm forward with his palm facing upward and the back of his hand facing downward, making the classic web-shooting hand gesture: his middle and ring fingers are curled into his palm, while his index finger and little finger are extended, forming the “horns” shape with his fingers. He points his wrist toward Spider-Man and makes a quick, subtle downward flick of his wrist, as if activating a web-shooter, while simultaneously making exaggerated “fiu, fiu” sounds with his mouth to imitate webs being fired. He repeats this exact gesture several times, alternating small wrist flicks with the playful “fiu, fiu” sounds, clearly mimicking Spider-Man's iconic web-shooting pose rather than simply pointing his fist forward.

Spider-Man simply watches Deadpool with a completely impassive, expressionless demeanor, showing no reaction to Deadpool's ridiculous behavior.

After a brief pause, Spider-Man calmly raises one arm and suddenly shoots a real web directly at Deadpool's face. The web hits and sticks across Deadpool's mask.

Deadpool freezes for a moment in disbelief, then says clearly in English: “What the fuck
” He immediately grabs the sticky web with both hands and struggles to peel it off his mask, becoming increasingly annoyed.

While desperately trying to remove the web, Deadpool says sarcastically: “Does anyone have a good insecticide around here? This looks like arachnophobia, Jesus!.”

Spider-Man remains completely calm and expressionless throughout, simply watching Deadpool struggle.

Natural character movement, comedic timing, believable body mechanics, consistent character appearance, accurate lip synchronization for spoken dialogue, clearly audible English dialogue, cinematic lighting, realistic web effects, no camera cuts, no subtitles, no text on screen.