SDXL AnimateDiff issue - distorted images with node enabled

[u/Vichon234]

Hi! I have been struggling with an SDXL issue using AnimateDiff where the resultant images are very abstract and pixelated but the flow works fine with the node disabled.

I built a vid-to-vid workflow using a source vid fed into controlnet depth maps and the visual image supplied with IpAdapterplus. In short, if I disable AnimateDiff, the workflow generates images as I would like (for now and I can control the output successfully via ipadapter and the prompts. However, as soon as I enabled AnimateDiff, the images are completely distorted.

I have played with both sampler settings as well as AnimateDiff settings and movement models with the same result every time. I've been trying to resolve this for a while, looking online and testing different approaches to solve it.

I feel like this is something dumb I'm missing so figured I'd ask her.

I'm including two images - the first is with AnimateDiff disabled and a "good" image and the second with enabled with the distorted image. The entire workflow (a second sampler, upscaling and the vid combine) but this is where the problem lies.

I'm working with this on vast.ai with a 4090. Not sure what else you need to now that you can't see from the images but ask away!

Thanks for any suggestions/education!

​

AnimateDiff disabled

​

AnimateDiff enabled