1:1 mentoring with Big Tech AI engineers
Q45Premium

When do you trust an LLM judge, and how do you know the judge is any good?

Evaluation & Quality

EvaluationLLM-as-JudgeQualityProduction

Asked at Scale AI · Anthropic · Sierra

How to Answer

“A judge is a model, so it has every failure mode a model has, plus one that’s worse: you stop reading outputs, because now you have a number.

So I treat it as a classifier that must be validated. Take a few hundred cases, have humans grade them, have the judge grade them, and measure agreement — Cohen’s kappa, not raw accuracy, because on a skewed set ‘always say pass’ scores ninety percent. Below about 0.6 the judge isn’t ready, and usually neither is the rubric.

The biases worth naming: position bias in pairwise comparisons, so swap the order and re-run. Verbosity bias, where longer answers score higher, so control for length. Self-preference, where a model favours its own family’s outputs, so don’t judge with the model that generated.

And the rubric matters more than the model. A binary criterion with a clear definition gets high agreement; ‘rate helpfulness one to ten’ gets noise. Then re-validate whenever the judge’s model version moves, because a judge drifting quietly is what invalidates months of results.”

The deep dive — diagrams, tradeoff tables, and the follow-up trap

Loading…