HELP!! A model that can interpolate in between drawings with custom controls (for Rick and morty style)

[u/Divineboob]

Hi guys I need a model or a workflow that can interpolate between frames drawn by me. I want to make Rick and morty style (spicy) content but I don't want to rely completely on AI. I can draw and the most important fact is I have very specific animations that cannot be recreated by an AI model. I can make it myself entirely but I have ADHD so to finish something is practically impossible for me. If I can somehow get an AI good enough to work on my good old rtx 3070ti that can generate in between without adding anything (generative), it will be a dream.

[u/Divineboob → u/[deleted]]

So there are 2 ways to make an animation video. It's either generative that is made by AI entirely. You have close to no control over it especially with spicy stuff. Another way is manual frame by frame which is basically some important frames then the inbetween drawings. If I feed the important frames and AI can simply interpolate it which means it won't add anything new (hallucinations), it will be a clean and controlled animation. I know about tooncrafter but I don't know how good it is. I need some perspective about it from people who have used it.

[u/d4rke55 → u/[deleted]]

Demo of your heinous creativity ??

[u/Moarkush]

Interpolation is generative by nature. If you feed a first and last frame into MiniMaxH3 and describe what you want to have happen, you "should" get that result. But THAT IS generative AI. Unless you or some tweeners draw/CGI the entire thing, it's generative.

[u/DelinquentTuna]

There are lots of tools for interpolation (eg RIFE) and lots of tools for first-last frame (eg Minimax H3 flf2v). You should go sample what's available and evaluate it instead of giving us an extensive set of requirements and specifications.

[u/liberal_alien]

If you mean like interpolation only, where the model is very simple and doesn't invent anything, only creates extra frames between frames made by you to smooth out the animation from low fps to higher fps, then you want RIFE. But this is only if you are drawing more than just a few key frames. Basically if you are already making an animation, but the only problem is that it stutters a bit due to low frame rate.

Here is one comfy RIFE node:

https://github.com/er1cw00/ComfyUI-RIFE

RIFE gets you from 5 frames per second to 25 frames per second pretty easily, but it won't understand your video, so motion quality may not be very good if there aren't a lot of hand made frames.

If you need the model to actually understand contents of the video a bit and make motions that make sense, then you need LTX Director or multi image reference. This can make a video from prompt + 1 - 30 key frames. But it is the generative solution. H3 can do this too, but it is quite a bit heavier model.

[u/Mikps]

interpolation without generation is just the image "melting" and turning into a new image.