Prompt Example, Not Training Data, Was Causing Fine-Tuned Model to Repeat Closing Lines

A developer running a fine-tuned Llama 3.3 70B model on Amazon Bedrock discovered that 36% of its generated story endings repeated the same grammatical template, initially suspecting overfitting in the training data. A quick count revealed the pattern appeared in only 0.3% of the 1,610 training examples, making a training-data explanation statistically impossible. The real cause turned out to be a single hardcoded example in the generation prompt itself, which the model treated as the dominant signal and consistently replicated. Rather than retraining at a cost of roughly $30 and five hours of compute, the developer replaced the one fixed example with a pool of seven structurally varied closing lines. The incident highlights how a single concrete example in a prompt can override fine-tuned model behavior entirely, making prompt auditing a critical first step before any retraining decision.