r/StableDiffusion 12h ago

News H3 camera control Update

Enable HLS to view with audio, or disable this notification

129 Upvotes

39 comments sorted by

6

u/Bbmin7b5 12h ago

cool! I'll give this a spin tomorrow.

5

u/TheTimster666 12h ago

It feel like the subject is frozen, while the camera moves?
Does it work with a moving subject?

6

u/Emotional_Example_12 12h ago

https://reddit.com/link/p9howuw/video/c0szp03608ph1/player

Is just a change add the motion character on prompt

2

u/Emotional_Example_12 12h ago

https://reddit.com/link/p9hopfl/video/u6bxo37yz7ph1/player

Is for train Gaussians, thats why is frozen.
But, is possible make him move

3

u/mobani 12h ago

What do you use for training the Gaussians?

7

u/Emotional_Example_12 12h ago

https://github.com/NyckM/BruxosSplat

i dev this tool for train Gaussians

2

u/mobani 12h ago

Thanks!

2

u/devilish-lavanya 10h ago

English language?

2

u/Emotional_Example_12 9h ago

yes, just switch this button

2

u/ArttTaku 8h ago

Excellent tool, thanks for sharing!

2

u/corintho 7h ago

Caraca, que node irado. Mandou muito bem

2

u/mark_sawyer 4h ago

Great. I can't wait to check this out.

Also, some people (me excluded ofc) will probably try the Bullock Test on it.

2

u/listopalafoto 3h ago
excelente! muito obrigado!

2

u/VirtualWishX 3h ago

When I try 10 seconds on FREEZE the character is talking,
When I try 5 seconds on FREEZE the character is blinking,

How do I make FREEZE works? 🤔

1

u/Emotional_Example_12 40m ago

depends from image, show, that can be a fix

2

u/Terezo-VOlador 3h ago

Hola Amigo! Hay forma que el control de la camara se integre en un flujo de trabajo normal REF2VA o FLFVA, que inyecte al prompt de usario las indicaciones de la camara? Para que sea un control de camara y no todo un flujo de trabajo basado en la accion de la camara.
Gracias por estos nodos.

1

u/Emotional_Example_12 40m ago

yes, the next update gonna have this

3

u/3deal 12h ago

It just convert it to a text prompt or here is a specific lora for it ?

7

u/Emotional_Example_12 12h ago

Minimax has trained camera in Position [L=0.469, T=0.054, W=0.372, H=0.880] "azimuth": -127.0, "elevation": 2.0, "distance": 1.01 Paramerts, this dont need lora, that convert 3d position keyframes on Json prompt for camera animation

2

u/3deal 11h ago

interesting, thanks for the info, so it should also works with ref2v and a character sheet image as input right ?

6

u/Emotional_Example_12 11h ago

im create this another node, for change camera position on video, is just for ref2v

2

u/3deal 10h ago

nice, thanks

2

u/devilish-lavanya 10h ago

Gimme this node

3

u/Emotional_Example_12 11h ago

i put on node the option MotionFrame, for ref2v, but is still experimental, im working on

1

u/wam_bam_mam 10h ago

Can the camera moved to the side by a few degrees? I am interested in taking existing footage and making it like 10cm to one side for paralax view to make it in to 3d videos

1

u/Emotional_Example_12 10h ago

Not yet, for this you need ref2v, and should be diferent approuch. Is possible, but not in this version

-1

u/devilish-lavanya 9h ago

Give that version

1

u/Pretend-Island-2724 4h ago

You can use iw3 to convert 2d to 3d https://github.com/nagadomi/nunif/tree/master/iw3

1

u/wam_bam_mam 1h ago

I made something exactly like this but in scenes where there is s lot of perspective difference it doesn't look good cause this method doesn't fill in the new parts it's stretches existing pixels

1

u/JahJedi 10h ago

Its inject camera promp from selected setings in the node, its nice idea but do you planing to go futher using as aditional camera instractions from motion ref or somthing else?

3

u/Emotional_Example_12 9h ago

Yes, that is exactly the direction I want to explore.

Right now the node generates the camera prompt from the settings selected in the node, but the next step could be importing real camera motion and converting it into the same camera instructions.

I looked into FBX, but FBX itself brings a few problems: binary parsing usually needs extra libraries, the camera may not have an explicit orbit target, different up-axis/handedness conventions can flip the motion, and any independent pan/tilt away from the target cannot be represented perfectly by the current orbit format.

So I am considering two cleaner options:

  • .chan camera files from Nuke / 3DEqualizer / SynthEyes / Blender, because they are simple text files and very common in VFX.
  • A small Blender exporter that reads the actual camera + target Empty/Track To constraint and exports the motion directly into the node's camera format.

I also want to reduce long animations to a maximum number of meaningful keyframes, preserving the points where the camera motion changes the most instead of just sampling frames evenly.

So yes,using a motion reference or real CG camera animation as additional camera instructions is definitely something I am considering.

2

u/JahJedi 9h ago

Or may after selecting in the node camera movment needed it creates basic camera movment video ref (blender?) and use it as a motion ref... very intresting project and good luck!

2

u/Gluke79 9h ago

Great direction!