Hung integration test blocked hotfix deploy for over three hours at checkout

·3 views

A one-line fix for a checkout bug that rejected a card type was merged in August but took nearly four hours to reach production. The delay was caused by a shared GitHub Actions runner labelled 'deploy' that was simultaneously running a slow integration test stuck in a blocking read. The test had connected to a message broker undergoing maintenance, which accepted the connection but never responded, and with no read timeout, no per-test limit, and no job timeout set, it inherited the platform's six-hour default. Engineers discovered the issue at 5:30 PM by checking what the runner was actually doing, and the hotfix deployed within nine minutes of cancelling the stuck test. The team has since separated deploy and test runners, enforced explicit job timeouts via linting, added per-test timeout limits, and set up on-call alerts for any deploy job queued longer than five minutes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How a RuneScape: Dragonwilds Guide Models Unverified Facts for Better Documentation

RuneScape: Dragonwilds reached version 1.0 on 15 September 2026, leaving early guides to navigate a game with more systems than anyone had yet confirmed. A dedicated guide site publishing 22 articles tackled this by explicitly marking the boundaries of each page's knowledge, using sections such as 'What this page cannot tell you yet' to distinguish verified claims from open questions. Individual pages also label the quality of their sources, noting, for example, when performance figures come from a single creator's video rather than official benchmarks. This approach treats unverified information as a third content state distinct from both confirmed facts and outright unknowns, rather than defaulting to silence or unqualified assertion. The site's patterns are presented as transferable practices for API references, runbooks, and internal technical handbooks facing similar gaps in available evidence.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Implements Command Pattern in JavaScript with Undo Functionality

A developer completed a JavaScript design patterns exercise by building a simple text editor using the Command Pattern. The implementation encapsulates editing actions — such as appending and deleting text — as discrete command objects, making them easier to execute and reverse. Four core components were used: a TextEditor receiver, AppendCommand and DeleteCommand classes, and an EditorInvoker that maintains a history stack. The invoker executes commands and enables undo operations by popping the most recent command from history and calling its undo method. Tests were written using Node.js's built-in assert module to verify the behavior of each component.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Design a Database for a Music Marketplace Like Discogs

A software developer and database author is documenting the logical database design process for a Discogs-style music marketplace in a multi-part series. The project models the site's three core layers: a massive content catalog of artists, albums, and releases; a social layer where users track ownership, wishlists, and reviews; and a peer-to-peer marketplace for buying and selling physical music. The design methodology follows an anchor-attribute-link framework drawn from the author's own database design book, keeping the schema database-server-agnostic at this stage. Using Pink Floyd as a worked example, the article walks through the complexity of modeling artists, release formats, labels, countries, and track credits. Physical schema details such as tables, indexes, and query optimization are deferred to later parts of the series.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Cloudflare Email Routing Lets You Forward Custom Domain Mail for Free

Cloudflare Email Routing allows users to forward emails from a custom domain address, such as hello@example.com, to an existing external inbox like Gmail at no cost on any plan. The setup requires the domain to be on Cloudflare DNS with no existing MX records from another mail host, as Cloudflare takes ownership of the MX configuration. Users must verify a destination address before creating routing rules, and Cloudflare automatically adds the necessary MX, SPF, and DKIM DNS records. Individual routing rules can be created for specific addresses, while a catch-all rule can handle or discard mail sent to unrecognized addresses at the domain. The service supports only inbound forwarding; sending email from the custom domain requires Cloudflare's separate Email Sending product, which remains in beta.

0 comments Read more at DEV Community