When two models, or two robots, disagree, at least one of them is wrong. That is useful: it tells you where not to trust an answer, without knowing the right one.
Background
- Query by committee (1992) asks for a label where a committee of models splits.
- Deep ensembles read uncertainty from the spread among independently trained networks.
- Self-consistency samples several reasoning chains from one LLM and trusts the answer most of them reach.
- Selective classification gives the frame for abstaining: trade how often you answer (coverage) against how often you are wrong when you do (risk).
- Calibration: without it, fusing confidences lets the most overconfident source win.
Robots
RoboTwin 2.0 has a VLM observer that watches the policy, and Agentic Robot has a verifier that checks the executor’s work. GoGet uses disagreement directly. It trusts the segmenter’s object count only when the segmenter and a vision-language model agree, and hedges when they don’t. On a robot, the hedge can turn into an action: move closer and look again, which is active perception driven by disagreement.
Resilient consensus uses the same logic: a robot discards the neighbours that disagree most with everyone else.
Correlated errors
The assumption underneath is that the models fail independently. They often don’t. Self-consistency samples share one model’s blind spots, a judge panel inherits its weakest member, and a verifier that is also a VLM fails on the same images as the executor. Worse, a spoofed input fed to every robot in a team produces agreement on something false, and that looks like confidence. A recent preprint on uncertainty in scene graphs makes the same point: detections from one model fail together and should not count as independent evidence.
Questions
- How do you measure how independent two models’ errors are, before trusting their agreement?
- When two robots disagree, which one should move, and which one should ask?
- Is disagreement between a camera and a lidar worth more than disagreement between two cameras running the same detector?
Related: evaluating language models, robust estimation, Byzantine fault tolerance, multi-robot multimodal perception.