RAGAS Framework
The four RAGAS metrics by hand and in code (0.4.3): what each compares, which need a reference, what a run costs, and what breaks on a fresh install today.
Last updated
After this section you can
- Map each RAGAS metric to the RAG gate it scores
- Run the 0.4.3 metrics with a Claude judge and check each result
- Count the judge calls in a run, and say which metrics need no reference
12
RAGAS
Map each RAGAS metric to the gate it scores, run it on RAGAS 0.4.3, and know what a run costs.
Key idea
Each RAGAS metric compares fields of one logged row, using a judge model (an LLM that grades text). Faithfulness scores gate 5, context precision gate 3, context recall gates 2 and 3, and answer relevancy none. No metric checks gates 1 or 4 directly.
Three of the four scores point at a gate; none checks gates 1 or 4 directly
Gates as in RAG Architecture: 1 index, 2 candidate, 3 cut, 4 prompt, 5 used and cited. Teal dots: the fields each score reads. Wider than your screen? Scroll sideways.