Misconfigured Kubernetes Health Probes Turned a 10-Minute DB Outage Into 40
A misconfiguration in a Spring Boot application's Kubernetes health probes caused a brief database outage to spiral into a 40-minute incident. Both the liveness and readiness probes were pointed at the same /actuator/health endpoint, which aggregates all dependency statuses including the database. When the database went down, Kubernetes interpreted the failed liveness check as a broken process and began killing and restarting pods in a continuous loop — even though restarts could not fix an external dependency issue. The correct approach is to separate the two probes: liveness should only reflect internal process health, while readiness should include dependency checks like the database. Spring Boot Actuator supports this split natively via health groups, allowing traffic to be paused during outages without triggering unnecessary pod restarts.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in