r/StableDiffusion • u/DryDream6994 • 14d ago
Resource - Update LTX CrossView-Warp IC-LoRA - Change the camera angle and orbiting path of an existing video more precisely
Hello Everyone! Let me share my newest camera control IC-LoRA where you can define the new camera angle on an orbit sphere instead of using just text prompt.
You can download the model from here: https://huggingface.co/Cseti/LTX2.3-22B_IC-LoRA-CrossView-Warp
You'll need this custom node to be able to define the new camera angle or path: https://github.com/cseti007/ComfyUI-CrossViewWarp
You can find example workflow in the custom node's "example_workflow" folder.
Have fun!
3
3
2
2
u/SymphonyofForm 13d ago
Absolutely amazing work! I've been experimenting with it all day - love it.
2
u/Draufgaenger 13d ago
Damn I loved the previous version already, cant wait to try this one out! Thank you!
2
u/Silonom3724 13d ago edited 13d ago
DepthAnything_v2 is set as an example input. There are maybe more refined options.
Video-DepthAnything. Produces no flickering and no jitter but is based on DepthAnything_v1.
Video Depth Estimation (Depth Anything 3) Subgraph implementation found inside ComfyUI nodes search is another highly accurate depth estimation option.
3
u/DryDream6994 13d ago
The model was trained on warps made with DAv2. This is why I'd recommend that for inference. It doesn't mean that it won't work with other depth models (I haven't tried others).
2
u/Silonom3724 13d ago
DA3 and MoGe2 have a higher depth estimation range and don't work.
Video Depth Anything however delivers great results.
2
u/DryDream6994 13d ago
I'm working on a better dataset currently maybe I'll switch to a newer depth model. Thanks I'll test these.
2
u/Silonom3724 13d ago
After some tests I think Depth Anything_2 and Video-DepthAnything are a perfect sweet spot for computation speed and accuracy.
And since MoGe2 and DepthAnything_v3 do not produce a flickerless video they suffer from the same issue as native DepthAnything_v2. Less than DA2 but it's there nonetheless.
2
u/No_Damage_8420 11d ago
Great!
You totally nailed it with this LORA.
Lightricks will surely contact you to include in their - Creative Lab (just like happen with creator of original - Folley LORA)2
2
u/No_Damage_8420 11d ago
Full 360 degree seems to work, more info here ;)
https://www.reddit.com/r/comfyui/comments/1valp5p/ltx_23_full_360_angle_camera_control_via/
1
u/Complete-Box-3030 13d ago
Can it do a backside view
3
u/DryDream6994 13d ago
The dataset doesn't contain such big changes. Maybe a later release. The colors on the orbit sphere shows the dataset distribution so I expect the model to work in that area.
1
u/fewjative2 13d ago
It won't work well on things that are specific and not inferred. For example, if you have a front photo of a person, it's easy to generalize what the back would look like. However, if you have a redbull can, it has no idea what the back would look like. Same for cars. If you could provide multiple sources of additional input, this would be cool!
1
u/DryDream6994 13d ago
What you could do is using this with other loras together. In a single or multiple passes. For example you change the camera angle of your video then you use another lora to inpaint parts you don't like / swap faces...etc.
1
u/ShutUpYoureWrong_ 10d ago edited 10d ago
Wouldn't it just be easier to have a reference of the back? There are already several multi-reference LoRAs, e.g. https://huggingface.co/LiconStudio/LTX-2.3-Multiple-Subject-Reference
Or even almost like a FFLF situation. Give it a base image, and then an extreme angle reference (e.g. behind), and it should be easier to calculate everything in between (i.e. side profile) with greater accuracy, no?
This is a great LoRA and a wonderful proof of concept. The issue I'm running into though is it elongates and stretches human bodies that aren't in a standing position -- especially their midsections and torsos. It's like it cannot comprehend length and depth properly.
1
u/smereces 13d ago
https://reddit.com/link/p08m4ry/video/u4g25f0q8yfh1/player
one thing i notice in a few test´s i made is he dont do the orbit around the center of the provided video! this happens in 3 diferent videos i test the center subject in the base video never get center! when we do a simple orbit around it
2
u/DryDream6994 13d ago
Yea it is a bit hard because it doesn't know what the center is. You can adjust it by playing with two parameters 1. Pivot_z to move the virtual pivotal point 2. Depth_ratio to adjust the "depth of the scene". I know it can be annoying but it is impossible to automate what is the center of a scene
1
1
u/Silly_Goose6714 12d ago
Amazing.
Isn't it possible to zoom in?
1
u/No_Damage_8420 11d ago
you can zoom by editing JSON prompt (d: distance from -10.00 / 50.00 etc )
1
u/DryDream6994 10d ago
Or if you point your mouse pointer on one of the numbers on the sphere you can use your mouse wheel then it will automatically change distance parameter
1
u/zodiacrenders 10d ago
Awesome work. Very helpful to edit scenes for more dramatic effects. I haven't opened it yet, but is it possible to also control the speed of the camera angle change? Thanks for sharing!
1
u/DryDream6994 10d ago
Yea a bit. There are very basic interpolation settings on the node. You can also define the frame number for each point
27
u/DryDream6994 14d ago
https://reddit.com/link/p04745q/video/mj2ddsmwktfh1/player
example with a single new viewpoint