Why LLM Code Review Verdicts Should Be Treated as Inputs, Not Final Answers

AI code review tools typically output hard labels like 'approve,' 'block,' or a severity score, which teams often treat as ground truth without questioning the uncertainty behind them. The core issue is that large language models are not built as calibrated classifiers — they are trained to produce plausible text, meaning their confidence scores carry little statistical reliability. The same bug can be flagged as 'critical' in one run and 'looks fine' in another with only a minor prompt change. One proposed fix is to feed the LLM verdict as an input feature into a small logistic regression model trained on a team's own historical data, allowing threshold tuning based on real outcomes. This approach enables per-severity calibration and adapts to a specific codebase's patterns rather than relying on the model's generic training distribution.