Celery Workers Can Leave Orphaned Subprocesses When Terminated, Article Explains

The article describes how a Celery worker shutting down after receiving SIGTERM can fail to pass the signal on to child subprocesses it spawned. Those children are then reparented to PID 1 and keep consuming CPU, memory, and resources. The author proposes a ZombiePreventionContext context manager that tracks and cleans up child processes during abrupt shutdowns. The approach is validated with a local simulation that sends SIGTERM mid-execution and checks cleanup. Recommended practices include using shell=False so signals reach the target process directly, and a staged cleanup that sends SIGTERM first and falls back to SIGKILL after a timeout.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in