RAG Architecture
What RAG is and when to use it instead of a long prompt, the offline and online pipelines, what Claude receives as cited sources, and five numbered gates that locate the stage which lost a wrong answer.
Last updated
After this section you can
- Choose between a cached long prompt, RAG and agent search for a given corpus, with the per-query input cost worked out
- Trace one question through the offline and online pipelines and say what Claude receives at the generate step
- Name the gate that lost a wrong answer and the first thing to inspect
RAG Architecture: Two Pipelines, Five Gates
RAG is a common way to give a model your own data. When to use it, how its two pipelines work, what Claude receives, and five gates that show where a wrong answer was lost.
A model knows nothing about your documents unless the right passages are in its prompt. RAG (retrieval-augmented generation) puts them there: an offline job indexes your documents, and each question searches that index and hands the best passages to the model. The right passage must clear five gates; when an answer is wrong, find the one it missed.
Gates: 1 in the index, whole and current · 2 a candidate · 3 survives the cut · 4 in the prompt, whole · 5 used and cited. On a phone the figure pans sideways.
Running example: Acme’s support docs (refund-policy.pdf, a payments runbook with error E-4502), 40,000 chunks, tenants A and B (separate customers). Numbers are illustrative.