SuperMimic trains one policy to track a whole library of human motion clips on a humanoid, and does it on a single GPU. Instead of running PPO against a simulator for days, it learns one differentiable world model of the robot’s dynamics from logged MuJoCo Warp transitions, then improves the policy by backpropagation through time over short rollouts imagined in that model. The model and the policy train together in one loop (Dyna-style): collect data with the current policy, fit the model on 8-step windows, then push the policy’s return gradient through 32 imagined steps, with a critic estimating the rest. Because every clip is performed by the same body with the same contacts, one model serves the whole library, and adding motions mostly adds evidence instead of new things to learn. The policy runs on a real Unitree G1.
GPU-hours to match the best published motion tracker
GPU-hours · log scale · lower is better
- SuperMimic3One H100 for 3 hours, already past SONIC's return
- SONIC21,000128 L40 GPUs for 7 days, PPO in simulation
Motions the tracker fails on, out of 129,514
One training iteration
The current policy runs in MuJoCo Warp with action noise. Transitions go into a replay buffer, newest replacing oldest.
An MLP learns to predict the next state, reward, observation and a fall probability, rolled 8 steps forward on logged actions so it learns to correct its own errors.
From states in the buffer, the policy acts for 32 steps inside the frozen world model, with sensor noise added back in.
The imagined return is differentiated through the model's Jacobians, an exact gradient instead of PPO's sampled estimate. A critic covers what happens after step 32.
Every iteration, so the model and the policy improve together
Findings
- On one H100, it passes the normalised return of SONIC’s released checkpoint within 3 hours. SONIC trained for 7 days on 128 L40 GPUs (about 21,000 GPU-hours), so this is roughly 56 times less wall-clock time and 7,200 times fewer GPU-hours.
- On a benchmark of 129,514 motions it fails on 25 after 3 hours on one GPU and on none after 9.8 hours on two, against 179 for SONIC.
- In a controlled comparison on LAFAN1 (40 clips) with the same reward, observations and networks, SuperMimic reaches a return of 0.82 in 1.2 hours on one GPU. PPO in the simulator never reaches it in 18 hours: 0.60 on one GPU, 0.70 on four and 0.72 on eight.
- The world model’s 8-step prediction error stays between 0.007 and 0.011 as the motion data grows from 1 to 264 hours, and the compute needed to reach the target return grows slowly.
- Where the gains come from: the model runs over 10 times faster than MuJoCo Warp at the batch sizes used (62 times at 64k environments), and the exact gradients reach PPO’s best return on the same world model with about 64 times fewer imagined steps.
- The critic matters. Without it, training fails at horizons of 16 steps or fewer. At matched return, the policy with a critic recovers from 0.5 to 4 m/s pushes within 0.5 to 2.4 s; the one without keeps stumbling and falls at 4 m/s.
- On hardware, one tracker does a skipping jump, a side roll, a 90° flip and an in-place 360° flip with no per-motion tuning.
- A velocity-command interface trained on its own reward through the world model beats a distilled one on 650 of 672 step commands. Only the distilled one falls, on seven fast turns.
Limits
- The world model learns only from the simulator, so whatever the simulator gets wrong about the real robot, the model inherits.
- Flat ground only: no terrain, stairs or objects. The clips that needed them were filtered out (129,514 kept of 142,220).
- The comparison with SONIC crosses GPU types and datasets. The authors argue the gap is too large for those to explain, which seems right, but the clean comparison is the LAFAN1 one.
- No seed variation is reported; the authors say training was stable across the seeds they ran.
- The world model is deterministic and the random pushes used in other trackers’ training were dropped, because the model can’t represent them. Push recovery is tested only in simulation.
- The downstream-interface study is called preliminary by the authors.
Relevance
Two things for this diary. First, the critic result is a small piece of evidence for failure recovery: a policy optimised only over the next 32 steps tracks motions just as well but can’t recover from a push, because the reward for recovering lies past its horizon. A nominal score doesn’t show the difference. Second, a cheap, accurate, differentiable model of a robot is also a model of what it should do next, which is what a runtime monitor needs; compare the risk-predicting world models in Feel-WM and risk-informed world models. Open question: when the policy is trained against a learned model, does a large gap between predicted and observed state on the real robot reliably signal a fault, or does it mostly flag sim-to-real error? The G1 is also the humanoid SIREN attacks, so an attack on its safety filter would apply to trackers like this one.
Links
- arXiv
- Project page
- Subfield: Robot learning, Control