QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction

3 min read

QuerySplat predicts a 3D Gaussian Splatting scene from a handful of unposed images in a single forward pass. Most feed-forward methods tie each Gaussian to a pixel ray; QuerySplat instead uses learnable queries as scene slots, each decoded into 64 Gaussians, and splits the decoder in two. Geometry queries attend to features from a frozen VGGT-Ω and output centres, scales and rotations; appearance queries attend to RGB and Plücker ray features and output opacity and colour. Cameras and the coordinate frame also come from VGGT-Ω, so no poses are needed at test time. Early in training, a Chamfer term towards VGM depth and an opacity floor keep the Gaussians from drifting or going transparent, and both fade out after 20K iterations.

This is a reconstruction paper. It has no scene graph, no semantics and no robot.

Findings

  • DL3DV, pose-free, 4 input views: 24.58 dB PSNR, against 23.27 dB for TokenGS (which needs poses) and 21.96 dB for the best pose-free baseline, YoNoSplat. With 2 views it reaches 21.39 dB and with 12 views 23.70 dB. The abstract quotes average gains of 2.30 dB over the best pose-free and 1.04 dB over the best posed baselines.
  • Not every metric is won: at 12 views its LPIPS (0.230) is worse than posed YoNoSplat (0.196).
  • Ablations at 4 views: dropping the appearance branch cuts PSNR from 22.90 to 18.64 dB, and merging both into one branch gives 20.92 dB. Swapping VGGT-Ω for plain VGGT gives a slightly higher 23.07 dB.
  • On one H200, inference takes under 1 s for up to 4 views and 1.89 s for 12. Peak memory stays near 10 GiB up to 12 views and reaches 40.77 GiB at 300 views. Training used 64 A800 GPUs.
  • 50 steps of optional test-time optimisation lift 12-view PSNR to 27.72 dB.

Limits

  • A fixed budget of 8,192 queries caps the number of Gaussians, so large or complex scenes are under-represented (the authors admit this). They suggest reconstructing overlapping regions and merging them, but do not try it.
  • Trained only on DL3DV and judged only on rendering metrics; geometric accuracy is not measured directly.
  • Static scenes, datacentre GPUs.

Relevance

For 3d-scene-graphs this belongs at the bottom of the stack: quick dense geometry and appearance from unposed images, which a graph builder would have to segment and abstract. The proposed “reconstruct regions, then align and merge” step is the same alignment problem robot teams face in collaborative SLAM, and the paper leaves it open.