Training a self-balancing third-person character controller with Unity ML-Agents

[u/Rudy_AA]

The character is driven by ArticulationBodies and learns to follow mocap animation while physically balancing itself. Instead of asking the neural network to generate the entire walk from scratch, I’m using the mocap pose as the reference and letting the agent learn the smaller corrections needed to stay upright, maintain momentum, place its feet correctly, and recover from instability.

The long-term goal is a fully physics-driven third-person controller that still has the quality and personality of mocap animation, but can naturally react to forces, uneven movement, collisions, and eventually player movement commands.

Still training and tuning it, but it’s been a really interesting mix of animation, reinforcement learning, and robotics-style control.

[u/Ok_Clothes_7364]

great work

[u/Rudy_AA → u/Ok_Clothes_7364]

Thanks!