AI models replicate old bugs when shown legacy code, study across 4 models finds
A developer ran a controlled experiment across 32 generations using four AI models — DeepSeek, Llama 4, GLM, and Mistral — to test whether existing repository code influences what models produce next. Two conditions were tested: one showing a codebase with 11 duplicated, bug-ridden search components, and another showing the cleaned-up version with a single shared component. When shown the legacy code, models hand-rolled new 190-line components 100% of the time; when shown the refactored codebase, they reused the shared component in every single run. The generated legacy-style components reproduced specific historical bugs, including broken mobile keyboard handling and hardcoded query-length values the project's own documentation explicitly warned against. Not a single one of the 32 runs crossed conditions, suggesting that the state of a repository — not just the instructions given — decisively shapes what AI coding assistants write.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in