Watchdog Script Hung on Network and WMI Calls, Forcing Timeout Fixes

A team running self-hosted CI runners built a Supervisor script with an independent Watchdog to detect and recover from hangs. The Watchdog itself froze when sending a Telegram alert after the host woke from sleep, because the OS-level TCP connection stalled and PowerShell's timeout was ignored. The team moved alerts into a background job with a 20-second timeout, then patched further failures like abandoned mutexes and GitHub API lag. Later, the Watchdog hung for 14 hours on a local WMI query, since the WMI service itself had stalled. The fix was to wrap that local call in a timed background process too, so every watchdog operation now has its own timeout.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in