Certifiable Object Pose Estimation: Foundations, Learning Models, and Self-Training

3 min read

C-3PO estimates an object’s pose from a partial depth point cloud and also says whether to believe the answer. The paper defines -correctness (the posed CAD model is within Hausdorff distance of the true one) and proves it can be checked with two certificates. Observable correctness (oc) asks whether the output agrees with the input points and the shape prior. Non-degeneracy (nd) asks whether the input was enough to pin down a unique pose at all. The model detects semantic keypoints, then a differentiable optimisation layer corrects them before registration. Self-training uses the oc certificate as the label filter: the network only learns from its own outputs that pass, so no annotated real data is needed.

Findings

  • When oc = 1 and nd = 1 the output is guaranteed -correct (Theorems 4 and 5). Theorem 18 gives the conditions (bounded noise, dense CAD model) under which their non-degeneracy check provably works. In tests, certified outputs score 100% ADD-S on every YCB object reported.
  • YCB sim-to-real (6 of 21 objects in the main table): a keypoint detector trained in simulation gets 0% to 34% ADD-S; adding the corrector lifts it to 38% to 92%; full C-3PO with self-training reaches 68% to 100%, close to the fully supervised detector trained on real data.
  • The fraction of observably correct outputs grows during self-training, “eventually reaching near 100% in many cases”.
  • Degeneracy is real: on ShapeNet cap and mug the oc check passes for 100% and 82% of instances, but ADD-S stays low until the nd check removes the cases where the visor or handle is hidden (only 32% and 22% then remain certified).
  • Baselines DeepGMR, PointNetLK, FPFH plus TEASER++ and EquiPose do poorly on these depth point clouds.

Limits

  • Needs a CAD model and annotated semantic keypoints per object; category-level keypoints are left open by the authors.
  • The certificate thresholds have to be set per object (oc was 0.0316 times the object diameter in one experiment), and the authors discuss tuning in an appendix.
  • The guarantee assumes the shape prior is right. A certified estimate of the wrong object is ruled out only because oc tends to fail on other objects, which they show empirically.
  • Certification can reject most of the data (22% certified for mug), so a robot using it must have something to do when nothing passes.

Relevance

C-3PO is a perception module that monitors itself at runtime, and the same approach could apply to runtime monitoring of other learned components in robust robotics. It builds on TEASER-style certification and is extended to category-agnostic shape in CRISP. Open question for multi-robot perception: can one robot’s certificate be checked by another robot with a different view?