Why a Green CI Dashboard Can Hide a Broken or Empty Pipeline

A developer running an unattended pipeline for several months discovered two silent failures that never triggered error alerts. In the first case, a hardcoded month in a URL-verification script caused publish checks to return 404 from September onward, yet the job still exited with code 0, leaving the blog empty while the dashboard showed green for a week. In the second case, an image job completed 43 of 45 files before failing on a temp-directory cleanup due to a Windows file-handle issue, causing all output to be discarded because downstream logic relied solely on the exit code. The author argues that exit codes carry only one bit of information and cannot distinguish between a successful run and one that silently produced nothing. To catch such failures, the piece recommends validating artifacts by checking existence, size, file count, freshness, and whether files were actually modified during the current run, alongside adaptive baseline tracking that learns only from healthy builds.