How to Answer
"Multi-layer:
- (1)Citation enforcement — every factual claim must cite a retrieved chunk. Claims without citations are flagged.
- (2)Claim verification — extract individual claims from the answer, check each against the source material. Score = supported claims / total claims.
- (3)Self-consistency — generate the answer 3 times with temperature > 0. If answers diverge significantly, confidence is low — flag for human review.
- (4)Programmatic checks — for numbers, dates, prices: verify against the source data directly. No LLM needed.
- (5)'I don't know' calibration — train the model to say 'I don't have enough information' when context is insufficient. Measure the rate and verify it's appropriate."