When should a developer consider fine-tuning a model versus using prompt engineering to achieve the desired behavior?
Select an answer to reveal the explanation.
Short Explanation
Start with prompt engineering—it’s faster, cheaper, and reversible. Only fine-tune when you need consistent behavior across many examples at scale and prompts can’t reliably deliver it.
Full Explanation
The standard recommendation is to start with prompt engineering before considering fine-tuning. Prompt engineering is fast, flexible, and costs no upfront training expense. Fine-tuning requires labeled training data, training time, and ongoing hosting of fine-tuned model weights. It’s most valuable when you need to bake in consistent formatting or style that’s impractical to describe in prompts, reduce token usage by eliminating lengthy few-shot examples, or improve performance on specialized domains where you have extensive training data. The option saying fine-tuning should always be tried first reverses the recommended workflow. The option saying the techniques are interchangeable is wrong because they have very different cost, complexity, and flexibility trade-offs. The option saying fine-tuning is only available for Haiku invents a false limitation on fine-tuning availability.