MapAnything is one feed-forward transformer that takes one or more images plus any optional geometry (intrinsics, poses, depth, partial reconstructions) and regresses metric 3D geometry and cameras. It factors the scene into per-view depth maps, local ray maps, camera poses and one metric scale factor that lifts local reconstructions into a consistent metric frame.
Findings
- One model covers uncalibrated structure from motion, calibrated multi-view stereo, monocular depth, camera localisation and depth completion, and matches or beats specialist feed-forward models (abstract).
Limits
- Abstract only so far. In-air training data.
Relevance
Optional inputs are the useful part for field robots: a vehicle with a pressure sensor, a DVL or a sparse sonar could pass whatever it has as a prompt, as Pow3R-SLAM does with RGB-D. For a team, partial reconstructions from a teammate are another possible input. See foundation models for robotics and collaborative SLAM.
Links
- arXiv · project page
- Subfield: Mapping and localisation, Perception