Prompt Engineering or Fine-Tuning? How to Know When to Switch

Teams building LLM applications typically begin with prompt engineering, which requires no infrastructure, is instantly reversible, and costs little compared to model training. Prompt engineering works by modifying the input to the model, while fine-tuning directly updates the model's weights to embed desired behaviour by default. Experts recommend staying in prompt-engineering mode until evaluation scores stop improving across three to four consecutive changes, signalling a performance plateau. Fine-tuning becomes viable only when the task is narrow, thousands of real labelled examples are available, and cost or latency constraints make a leaner model preferable. Common warning signs that prompting has hit its limit include diminishing eval gains, conflicting instructions, bloated system prompts, and the same corrections being resent on every API call.