Tracking flaky Playwright tests across runs to tell flakiness from real breakage
Teams often re-run failing Playwright tests in CI until they pass, which lets unreliable tests pile up and erodes trust in red builds. Playwright's built-in flaky status with retries only covers a single run, so retries can hide problems and a re-run produces a new report that discards earlier failures. Reliable detection requires examining the same test across many runs, looking for passing and failing on the same commit, needing retries to pass, or repeatedly flipping between pass and fail. A test that keeps failing is a broken test needing an immediate fix, which differs from a flaky one that should be stabilized. Identifying a test across runs depends on its spec file, full title path, and project name, so renames or moves can reset its history.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in