Fine-Tuning a Small Safety Model for Coding Agents: Three Runs and Two Setbacks

A developer built laya-guard, a tool that sits between a coding agent and the shell to screen commands before execution. Simple rules handle clearly dangerous and clearly routine commands, while a 322M-parameter local model judges the rest. The first fine-tuning run was overly strict, flagging many everyday commands, while the second overcorrected and let several harmful commands through. The author traced each failure to gaps in the training data, including missing Windows commands, comments used as a signal of safety, and little coverage of credential leaks. A third run was needed to balance these trade-offs.