A Silent Cron Failure Went Unnoticed for 10 Weeks, Then the Watchdog Broke Too
A scheduled cron job in a GitHub repository stopped firing on 21 June 2026 and went undetected for ten weeks, causing scheduled blog posts to silently fail to publish. The team built a watchdog workflow to monitor scheduled jobs and alert via issues, and it worked correctly on its first run, filing an accurate report about the stalled cron. However, on the following three nights the watchdog itself crashed before completing its checks, producing failure logs visually identical to the successful alert run. The root cause was that GitHub's workflows API began returning a synthetic, file-less entry — "dynamic/dependabot/update-graph" — after Dependabot was enabled on the repository. When the watchdog tried to fetch this non-existent file via the contents API, it received a 404 error that killed the script entirely, leaving the actual broken cron unmonitored and the failure mode once again invisible.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in