How a Kubernetes Node Thrashed Silently While Every Health Check Showed Green
On August 17th, an engineer's new alert rules flagged two separate IO pressure incidents on a four-core ARM server, neither of which triggered any error from Kubernetes, ArgoCD, or Helm. Investigation revealed the heaviest disk reader on the node was a restarted Jaeger container — the third victim of OOM kills in 16 hours — thrashing the page cache rather than actively requesting reads. The node's memory limits were set at 110 percent of capacity, while the scheduler only reads memory requests, which stood at just 41 percent, giving it no reason to stop placing pods. Each layer of the stack answered its own narrow question correctly, but the composite picture they painted was entirely misleading. The incidents illustrate how overcommitted resources and siloed health signals can mask systemic instability until low-level kernel metrics are examined directly.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in