LoRA Adapters Showed No General Forgetting in Test, Until a Seed Control Gap Was Found
A researcher tested how much a LoRA fine-tune on a narrow legal classification task erodes a small language model's general knowledge. Across nine configurations of adapter rank and training steps on Qwen2.5-3B-Instruct, general-task retention never fell below the unmodified baseline, contrary to the registered hypothesis. Checks for parsing errors and reload inconsistencies found no artifact explaining the pattern. The real flaw was a missing random-seed control, and retraining the extreme configurations under new seeds showed the apparent gain was partly coincidental. The author reports that most configurations still delivered meaningful domain accuracy gains while keeping general retention above 95%.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in