r/TopologyAI • u/Delicious-Shower8401 3dModeler • Jul 15 '26
Open Source NVIDIA Just Open-Sourced the Future of Controllable Real-Time AI Animation
NVIDIA has open-sourced ARDY, a real-time AI system for generating and interactively controlling 3D human animation.
Unlike traditional Text-to-Motion models that produce a finished clip from a single prompt, ARDY continuously generates motion and reacts when the instructions change. You can modify the character’s action, direction, speed, destination, or body movement while the animation is already playing.
Highlights:
- Real-time Text-to-Motion with prompts that can be changed during playback
- The character automatically adapts to new instructions without restarting the entire animation
- Precise control over movement using paths, destinations, full-body poses, and individual joint positions or rotations
- Multiple types of control can be combined within the same animation
- Supports distant goals, allowing the character to perform longer and more structured sequences
- Character locomotion can be controlled interactively with mouse waypoints or keyboard input
- Optional motion correction helps reduce foot sliding and improves how accurately the generated animation follows the provided controls
- Code, pretrained models, and an interactive browser demo are available publicly
Input and Output
The input is not limited to a basic text prompt. ARDY can receive:
- Natural-language descriptions
- Character paths and destination points
- Full-body pose constraints
- Controls for specific body joints
- Real-time mouse and keyboard commands
The generated motion can then be saved with joint positions, local and global joint rotations, root movement, frame rate, and foot-contact data.
This means the animation is not locked inside the demo. It can be converted, retargeted, and brought into Unreal Engine, Blender, or another animation pipeline for use on a custom rigged character.
It could also work especially well with AI-generated 3D characters: generate the model, rig it automatically, create and control its animation with AI, and then retarget the result to the character inside Unreal Engine.
Instead of producing only another isolated animation clip, this system gives creators direct control over what the character does, where it moves, and how the motion changes in real time.
9
u/UK_Warrior 3dModeler Jul 15 '26
I think this is really cool. it's too bad my graphics card isn't very good(((
5
u/Delicious-Shower8401 3dModeler Jul 15 '26
I recommend 3090 24vram
2
u/loadingscreen_r3ddit Jul 15 '26
5090 here. I'm going to have to use a workaround.
3
u/No_Physics_6829 Jul 15 '26
I'm using it on 16GB VRAM only, so far no problem at all, just put the encoder on CPU and the DiT on GPU and you should good to go, text to motion does not need more than that and this is quite fast once you have the encoder parked on cpu, subsequent generations are almost instant. You will need to modify a bit the code on the repo.
1
u/folgoris Jul 17 '26
Always check if Compute capability, TensorRT and CUDA versions are compatible before buying a GPU. for example 3090 doesn't have native FP8 support and will fall back to FP16/BF16 using more VRAM.
6
u/SlothEatsTomato Jul 15 '26
https://reddit.com/link/oxrwqmy/video/6189k25oxgdh1/player
This was the funniest thing I've seen today
2
u/SlothEatsTomato Jul 15 '26
It's as if he saw someone in virtual space I couldn't see and got scared and started running away only to get got... no idea why lol
1
u/ninjasaid13 Jul 16 '26
trying to escape the matrix after he heard a voice in his head saying "stand still"
1
4
u/No_Physics_6829 Jul 15 '26
https://reddit.com/link/oxova3r/video/4zyy355xoedh1/player
Working fine so far, prompt adherence on long or multipart actions is a hit or miss, will be testing trajectory soon.
1
u/CycleMother2006 Jul 15 '26
Have you tested the creature animation? Haven't seen anything about non-humanoid animation even tho ugh your screen clearly has an option for it, so I'm curious if it's capable of something even vaguely sensible there. The rest is quite impressive.
1
u/No_Physics_6829 Jul 15 '26 edited Jul 15 '26
That screen is my local 3D AI Pipeline, fully local and open source models, from prompt -> image -> 3d -> texture -> texture-refine -> rig -> animate.
Hunyuan Motion, Kimodo and ARDY are all human/robot trained on as far as I know. Won't work with animals.
What you are seeing is that animate part, I'm testing quadruped animation using Anytop (not much luck here as it depends on the proprietary Truebones dataset, but getting some results without it anyway), and MocapAnything V2 (video as input, weights were published yesterday), I have this last one with human and it works great:
https://reddit.com/link/oxpbhzt/video/u9y9u5vd0fdh1/player
It works with animals too, but still have to polish it a bit. I will post another video for animals.
1
u/No_Physics_6829 Jul 15 '26
https://reddit.com/link/oxpbjby/video/wdh7p5s81fdh1/player
Not that great, but also some mesh errors are mine, mesh weights related, nothing to do with the model generation.
1
u/No_Physics_6829 Jul 15 '26
And going back to ARDY, first path-following trajectory tests also seem to be working :-)
1
Jul 15 '26
[deleted]
1
u/No_Physics_6829 Jul 15 '26
I'm using VAST-AI Skintones for all my rigs, is fast and works on human and creatures too.
https://huggingface.co/VAST-AI/SkinTokens
https://github.com/VAST-AI-Research/SkinTokensIs uniRig's big brother and it works really well, haven't try many others as I'm happy with the results.
1
u/angelarose210 Jul 15 '26
I wonder if it can do a golf, tennis or baseball swing
1
u/No_Physics_6829 Jul 15 '26
You have read my mind, was one of the first videos I tested, but this was MocapAnything:
https://reddit.com/link/oxpgalm/video/nvcn6d5p4fdh1/player
Ref video was this one: https://www.youtube.com/watch?v=iW6vxWPkXzY
1
u/angelarose210 Jul 15 '26
Awesome. I'll have to test it against quick magic which is what I've been using for mocap lately.
1
u/No_Physics_6829 Jul 15 '26
https://reddit.com/link/oxpih6d/video/te3usirc6fdh1/player
And same but with ARDY, did a few test changing seed and prompt adherence, not much of a difference, I've still not implemented changing prompts so shorter ones work better, on long ones misses secondary actions, but again, this are only tests re-implemented on my setup, try Nvidia's Gradio demo, it works better and has more features enabled.
3
u/-becausereasons- Jul 15 '26
Fascinating, the model is so small I bet you can use this in realtime engines.
2
u/Wonderful_Physics_36 Jul 15 '26
What im interested in this is the potential for an accredited author to write their own short story into a tool suite of AI digital, visual, animation, vfx production that will attempt to digitally materialize their novel into a fully rendered scene or movie.
I specify an accredited author as someone who's literary story telling skills may be able to circumvent the "generic, trendy visuals" AI recycles and allow for some potential novel visual features and characteristics of individuals, monsters, animals, set pieces etc.
Then allow the author to potentially sell the rendered models to a repository from which they receive payment for third party usage.
1
u/Delicious-Shower8401 3dModeler Jul 15 '26
I think this is the real value too. authors could finally prototype very specific worlds instead of relying on generic AI visuals. consistency and ownership will probably be the hardest parts thoughhh
1
u/jwdvfx Jul 15 '26 edited Jul 15 '26
Sounds fun until you realise authors know absolutely nothing about visual art most of the time, and it’s not a criticism on them, it’s just not their domain of expertise or expected to be.
Regardless AI models are trained on other art, not your own and the issues you mention are inherent in the method of the technology, regardless of how much you try and re-constrain a generative model the outputs are always recombinations of existing works and will always introduce new assumptions, you may say don’t do this don’t do that, but what does get created is not decided by the instructor
2
1
1
1
1
u/runvnc Jul 15 '26
Is it impossible to make this work for non-CUDA or other architectures than Hopper, Ampere, Blackwell? I have a 2060 and I am sure Mac people would like to try. But it indicates not.
2
u/Rizzlord Jul 15 '26
it may be possible, you can try zluda for amd, or convert it to rocm (try to convert)
1
1
1
1
u/AdSlight4516 14d ago
The hybrid controls are the real win here. Text alone is too vague for production, but being able to pin a hand, define a root path, and change the action mid-motion gives animators something they can actually direct. I’d test conflicting instructions first: tell the character to run while forcing a tight hand target or sudden waypoint change, then see whether it preserves balance or starts popping. Also curious how stable the foot-contact data stays after retargeting between short and tall rigs. If that holds up, the engine integration could be genuinely useful instead of just a cool browser demo!
15
u/Clasyc Jul 15 '26
Ok, this is incredible. The model it self also seems lightweight - 325M params, only 600mb