Robots now call language models to plan, to describe what they see, and to judge whether a task succeeded. Each of those needs some way to tell a good answer from a bad one, and the cheap way, asking another LLM, has known failure modes.
LLM judges
The LLM-as-a-Judge survey lists the biases: position (preferring the first answer), length, self-preference, authority and names. The attacks are cheap: a constant “null” answer, or starting a response with “Thought process:”. In the survey’s own experiment:
- Agreement with human labels barely varies across judges, from 50.72% to 61.54%.
- After the two answers are swapped, LLaMA3-8B keeps its verdict only 38.85% of the time.
- Asking the judge to explain itself made position consistency worse. Asking it to check itself did nothing. A majority over repeated runs helped.
- A three-model panel was only as good as its weakest member, whose bias dragged the vote.
Judges as rewards
CaRR shows what happens at training time. Rewarding a search agent only for the final answer let it learn shortcuts: it cited fewer pages and checked less. The fix rewards citations, but it depends on a judge LLM that was checked on 10 trajectories, and the benchmarks are scored by yet another judge.
Human graders
LalaEval uses human graders and adds a dispute analysis that flags graders and questions with unusual disagreement. The survey’s advice (a strong judge, plus swaps and voting) averages disagreement away as noise. LalaEval uses it to find unreliable graders and questions, which is closer to what a robot team needs; see disagreement between models.
Questions
- A robot that asks a VLM “did the grasp work?” is using an LLM judge. How often does it say yes when it shouldn’t?
- If a team’s robots all run the same model, is their agreement worth anything?
Related: foundation models for robotics, runtime monitoring, human-robot trust.