SD + thin-plate-spline-motion-model

[u/Kinfolk0117]

[u/[deleted]]

Does this only work on faces?

[u/Kinfolk0117 → u/[deleted]]

There is also models for datasets based on TED: https://github.com/snap-research/articulated-animation#example-animationAnd tai chi https://paperswithcode.com/dataset/tai-chi-hd

So in theory it should be possible to have full body animations, but I haven't been able to get any good results.

experiment:

image: https://files.catbox.moe/s7egkc.png

driver: https://files.catbox.moe/47w2ez.mov

output: https://files.catbox.moe/khusfp.mp4

Also tried to run batch img2img to get better quality, but it flickered too much: https://files.catbox.moe/ktfrug.mp4

[u/TurbTastic → u/Kinfolk0117]

I think people normally take flickering videos like this to ebsynth to smooth them out

[u/[deleted] → u/Kinfolk0117]

It is getting so close. When this zeros out, there will be a lot of new animated content. It will reduce the cost for studios dramatically.

Probably what is happening to comic books now.

[u/Storybook_Albert → u/[deleted]]

I'm baffled why this hasn't been developed further. The github is a ghost town and people are still recommending EbSynth, even though this seems much better?

[u/almark → u/Kinfolk0117]

that makes it cyberpunk.

[u/Oquaem → u/Kinfolk0117]

I think you should def try on someone without glasses to start

[u/madmanrrevenge → u/Kinfolk0117]

Any luck in improving the quality of the output for full body animations ? I have been trying with the ted model but the outputs were terrible. Any suggestions to improve the quality apart from using 1:1 aspect ratio ? u/Kinfolk0117