Six GnuTLS CI test jobs can fail without breaking the pipeline
A static analysis of GnuTLS's GitLab CI configuration found that six of its fifteen test-running jobs are set to allow failure, so a failing test still shows the pipeline as passed. The affected jobs include Windows (mingw) builds under Wine, a ThreadSanitizer suite, and several cross-architecture runs under QEMU. The project's other nine test jobs do hard-fail the pipeline, which suggests the exceptions were not intended to be permanent. Two of the jobs carry a comment about a single step failing in a rootless container, but allow_failure applies to the whole job, so real regressions could go unnoticed. The four cross-architecture jobs have no stated reason at all, and the big-endian ppc64le and s390x jobs are the only ones testing those architectures. The article proposes forgiving only infrastructure errors while still failing on genuine assertion failures.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in