Developer finds AI image scoring model returned identical scores across 40 images

A developer running a local AI-based quality gate for batch image generation discovered that their vision model was assigning nearly identical scores across all 40 images in a slate, with humor rated exactly 7 every single time. The flaw only became apparent after examining raw per-axis scores rather than relying on the composite pass/fail output, which appeared to vary slightly due to background scoring. Rewriting the scoring rubric with stricter numeric anchors shifted the flat axis from humor to wit, but did not resolve the underlying problem. Testing three different models on the same images revealed that only the largest model, qwen3-vl:32b-thinking, used the full scoring range and also correctly flagged four images containing garbled or mangled text. The developer concluded that checking the standard deviation of each scoring axis is a simple but essential step to verify whether an AI judge is actually measuring anything.