MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision

2 min read

MoGe predicts a 3D point map from a single image, up to an unknown global scale and shift (an affine-invariant point map). Leaving scale and shift free removes an ambiguity from the training signal. Two training tools carry the method: a fast solver that optimally aligns the predicted and true point clouds for the global loss, and a multi-scale local geometry loss for fine shape. From the same forward pass it reads off depth, the camera’s field of view and, per the README, normals.

Findings

  • The abstract reports that it beats prior methods on unseen datasets for monocular point maps, depth and field of view. It gives no single headline number.
  • The README quotes about 60 ms per image on an A100 or RTX 3090 in FP16 with the ViT-L model.

Limits

  • I read the abstract and README only.
  • No metric scale in this first version; MoGe-2 adds it.
  • Trained on in-air data. How it behaves when colour and contrast fall off with range underwater is untested in what I read.

Relevance

An estimated field of view helps robots running cheap cameras behind housings, where intrinsics drift. Scale-free geometry also pairs naturally with an external scale source such as a pressure sensor or DVL. See foundation models for robotics and MoGe-3.