RAG Evaluation
Build a golden set, compute recall@k, MRR and nDCG, decide whether a reranker earns its latency, score answers against their evidence, and read the scores together to name the gate that failed.
Last updated
After this section you can
- Build a golden case with source spans, grades, an answerable flag and a corpus version.
- Compute recall@k, MRR and nDCG on one ranked list, and choose the metric each question needs.
- Name the gate that lost an answer from the pattern of scores, and say whether a gain is noise.
11
RAG Evaluation
Build one golden set, score each gate on it, and read the scores together to name the gate that lost.
Key idea
Score each gate on one golden set, with the metric that fits it. Read the five scores side by side: the first failed check names the gate, and an average hides it.
The first failed check names the gate
Wider than your screen? Scroll sideways. Case B fails at gate 2: nothing after the pool can rescue a chunk that never reached it.