How a Passing Health Check Hid a Silent, Broken Background Worker

A software developer spent nearly 48 hours debugging a Python service after a deployment appeared successful but a critical background worker had silently stopped functioning. The HTTP process correctly responded with a 200 status on the /health endpoint, but this probe had no visibility into whether the separate worker process was actually running or draining its queue. Because the health check always returned a passing response, the developer repeatedly investigated the wrong areas — restarting the web process and checking environment configs — instead of examining the worker directly. The core problem was that the smoke test and health probe were only verifying network connectivity to one process, not the operational status of the full service. The incident highlights that a health check incapable of returning a failure provides no real diagnostic value and can actively mislead engineers during an outage.