Pilot Two Repos Before Fleet Rollout to Catch Shared Tool Failures Early

A developer maintaining roughly a dozen products built on a shared framework created a conformance linter and chose to test it in just two repositories before a full rollout. The two-repo pilot immediately revealed a fatal exit-255 error in CI caused by the linter loading the framework's own dependencies, which were unavailable in consumer environments due to shallow cloning. A second issue emerged when a linter rule incorrectly flagged a safe, guarded dev-secret pattern as a violation, producing false positives across nearly every product. Had the developer rolled out to all repositories at once, both failures would have triggered simultaneously, making triage far harder and likely undermining trust in the tool entirely. The author recommends selecting pilot repositories that maximize coverage of different consumption modes rather than choosing similar ones, to surface the broadest range of real-world failures cheaply.