Fine-Tuning an 8B Model on 250 Security Examples Degraded Key Reasoning Skills

A small-scale experiment fine-tuned an 8-billion-parameter language model on fewer than 250 cybersecurity classification examples using consumer hardware, aiming to improve its ability to identify threat categories. While the fine-tuned model became more accurate at labeling threat types, it performed worse on harder tasks such as severity scoring and lifecycle-depth counting. The degradation aligns with established research on 'jagged' capability profiles, where narrow fine-tuning sharpens one skill while weakening adjacent reasoning abilities, an effect shown to be more pronounced in smaller models. The experiment also observed a scope-collapse failure, where the model incorrectly routed an unrelated weather query through its full security-analysis pipeline instead of declining it. These findings reflect known phenomena — including catastrophic forgetting and fine-tuning-induced behavioral drift — documented in recent 2025–2026 studies on supervised fine-tuning of small language models in specialized domains.