How to Answer
“I’d start from the budget and work backwards, because at a tight enough number the budget picks the architecture. Say twenty cents a claim at fifty thousand claims a month.
At twenty cents I can’t put every claim through a frontier model with a large context. So: a triage step on a small model classifies the claim and retrieves only the policy sections that apply. Keeping retrieval tight is the biggest token lever, because retrieved text is what actually fills the prompt.
Then a decision tier. Clean claims that match a policy rule with high confidence get auto-approved by the small model. Ambiguous ones escalate to the frontier model with full context. Anything the frontier model isn’t confident about, or anything above a dollar threshold, goes to a human.
And then I’d point out that the economics live in that distribution, not in the prompts. Human review is the expensive line by an order of magnitude — if I want the cost down, I move the escalation rate, not the token count.”