Machine-Learnt Physically Based Animation

See the code for the agent here:

This project explores active ragdoll animation with machine learning. The original goal was to create a character that could adapt its locomotion to player input, changing speed, direction and gait while staying physically grounded.

The first dog-based pass was valuable because it exposed the hardest parts of the problem early. It established a workable baseline, revealed where the constraints were coming from, and informed a more practical second iteration.

• The original dog rig was tall relative to its footprint, which made balance materially harder to learn.
• Using IK to calculate joint angles simplified training because the network only had to choose foot targets, but it also constrained the range of leg motion available to the agent.
• Matching recognisable animal locomotion would likely require a much richer physical model of anatomy and joint behaviour, so a fictional creature became the more practical route for iteration.
• That first pass reached a useful plateau and clarified where deeper reinforcement-learning research would have the biggest payoff.

Using that first iteration as a baseline, I then made the following changes:

• Lowered the character's centre of mass relative to its height, which made balance easier to maintain.
• Switched to a fictional creature so unusual motion could be judged on readability and control rather than strict realism.
• Spent time improving my understanding of the training setup, reward structure and network tuning. That work had a clear impact on the quality and consistency of the resulting behaviour.

The current result is a creature that can walk and climb reliably. Reaching that point required improvements to the training manager, reward structure, environment design and difficulty-scaling setup, alongside a much stronger grasp of how the underlying learning process behaved.

Jumping remained the least mature behaviour. The agent could occasionally achieve it, but not yet with the consistency needed to count it as solved.

My working theory is that jumping is harder here because the agent depends on velocity and angular velocity readings that become less trustworthy during higher-force interactions such as take-off and landing. Those signals were good enough for walking and climbing, especially because the agent could grab and stabilise itself, but jumping appears to need more precise physical feedback. Unity's non-deterministic physics adds another layer of difficulty on top of that.

A likely next step would be to feed ground-reaction and joint-force data into the network so it has a better picture of what is happening during take-off and landing. Because the character uses articulation bodies rather than rigidbodies, those signals are relatively accessible.

At that stage, further gains were possible but expensive: each engineering change required a few hours of implementation followed by roughly ten or more hours of training and evaluation. I chose to pause the project once it had delivered its main research value - a working walk-and-climb controller, stronger ML intuition, and a clear picture of the technical hurdles behind more advanced locomotion.