When to Fine-Tune: The Decision Framework
Should you fine-tune at all? A structured decision framework for prompt engineering vs RAG vs fine-tuning.
Last updated
After this section you can
- Answer the three questions that decide whether fine-tuning is the right tool at all
- Compare fine-tuning, RAG, and prompting on the axes that actually differ
- Recognise the green lights and red flags before committing a training budget
When to Fine-Tune: The Decision Framework
Fine-tuning is the last tool you should reach for, not the first. The optimization hierarchy, the three questions that settle it, and the honest defaults — so you spend weeks of engineering only when the cheaper levers are genuinely exhausted.
“Should we fine-tune?” is the most common fine-tuning question, and the right answer is usually no. Fine-tuning changes model weights — it is the most powerful, most expensive, least reversible, and slowest-to-iterate option on the table. Almost every problem people reach for it to solve is actually a prompt problem (behavior) or a knowledge problem (facts), and both have cheaper, faster answers. This section is the filter: climb the optimization hierarchy and only fine-tune when the rungs below it have run out. The other five sections in this part are the how; this one is the whether.