How an AI Code Review Service Stayed 'Healthy' While Becoming Unusable

An internal AI-assisted code review service saw its p95 turnaround time spike from 41 seconds to over 3 minutes in under 20 minutes, despite no crashes, pod restarts, or upstream errors. The root cause was an admission failure: every incoming coding task was accepted into a single FIFO queue regardless of depth, causing head-of-line blocking when the upstream model slowed down. Engineers determined the fix was not scaling up workers — which were already idle waiting on the model — but rejecting new work before the queue itself became the outage. The team built a lightweight admission proxy that monitors queue age and in-flight task count, returning HTTP 429 responses with a Retry-After header once thresholds are breached. The resulting runbook includes a fault-injection harness, alert thresholds, and a rollback path designed so that removing the proxy fully restores prior behavior.