π0 puts a flow matching action head on a pretrained vision-language model, so the policy inherits web-scale semantics and outputs continuous actions. It is trained on a large mixed dataset from several dexterous platforms: single-arm robots, dual-arm robots and mobile manipulators.
Findings
- Evaluated on zero-shot performance after pretraining, on following language instructions from people and from a high-level VLM policy, and on learning new skills by fine-tuning.
- Example tasks: folding laundry, cleaning a table, assembling boxes.
- The abstract gives no numbers.
Limits
- Abstract-level reading only.
Relevance
In RoboTwin 2.0, fine-tuned π0 averages 46.4% on clean tasks and 16.3% under domain randomisation, the best of the policies tried but still a large drop. The same VLM-plus-action-head design means a misread scene goes straight to the motors. Available in LeRobot; see foundation-models-for-robotics and robust robotics.
Links
- arXiv
- Subfield: Robot learning, Perception