Study: 13 of 14 AI Models Produce More Complex Code Than Human Developers

A study by the team behind Python analyzer pyscn tested 14 AI models against 500 real bugs from SWE-bench, an open-source benchmark of closed Python issues with verified human fixes. Researchers measured changes in cyclomatic complexity and dead code before and after each patch was applied, comparing AI-generated fixes directly to the human commits that resolved the same bugs. Thirteen of the fourteen models regressed on code quality more often than the human developer who fixed the identical bug, while one tied and none performed better. The probability of this 13-out-of-14 outcome occurring by chance is approximately 0.00024, and twelve models individually cleared a p-value threshold of 0.05. Only patches that passed their respective test suites were included, and all result sets have been committed publicly for independent verification.