LLM code review tool flags false positives and misses critical bugs, study finds.

A development team integrated an LLM-powered tool, Claude via AWS Bedrock, to automate code reviews for a Java microservices project. The tool effectively identified issues like dead code and inconsistent logging. However, it also flagged many good patterns as errors and missed critical bugs like an important serialization error. The team concluded LLMs are not a replacement for human reviewers but act as a filter for a different set of problems. The process requires continuous prompt tuning to manage a growing list of false positives.