SShortSingh.
Back to feed

Automated cloud deployment pipeline erases emergency fix, causing repeated outage

0
·1 views

A company's automated cloud infrastructure pipeline runs daily at 7 a.m. to enforce network rules defined in code. An engineer manually added firewall rules at 3 a.m. to fix a partner's server migration issue. The morning pipeline detected the manual rules as unauthorized deviations and removed them, causing the same outage to recur. This highlighted a conflict between emergency manual intervention and automated configuration enforcement. The company has since updated its process to require immediate code updates for emergency fixes and added pipeline safeguards.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Git, Docker, and IPFS share foundational concept of content-addressing

Git, Docker, and IPFS are built on the principle of content-addressing, an idea central to modern computing tools. This system identifies data not by its storage location but by a unique cryptographic hash of its content. The hash acts as a verifiable fingerprint, ensuring data integrity by changing completely if the data is altered. This foundational concept provides inherent security and consistency across different applications.

0
ProgrammingDEV Community ·

Developer details pixel-perfect UI challenge for 372 test cards

A developer described a technical challenge involving the precise visual rendering of 372 digital practice test cards. The issue centered on a one-pixel rounding gap that became visible at a 3% zoom level. The problem was solved by applying a negative one-pixel inset to an image wrapper, allowing the image to overhang its container. This fix differed from a simpler scaling solution, which would have proportionally cropped content on a 373rd card type. The component was ultimately generalized to handle two distinct content formats with different layout requirements.

0
ProgrammingDEV Community ·

Munchable app shows legal disclaimers diverge across codebase despite requirements

Munchable, a food information app for people with gut conditions, must display specific, legally-mandated disclaimer sentences. An internal review found four instances of the core required disclaimer, with three showing variations in wording. The discrepancies exist between the main rules-engine package, the website's API copy, and the website's footer component. This highlights a challenge in maintaining consistent legal language across different parts of a software monorepo.

0
ProgrammingDEV Community ·

Simulation Shows Supply Chain Cuts Only Stop Wars Under Specific Conditions

A developer created a mathematical simulation to test whether cutting critical supply chains can halt a military conflict. The model pits stockpile depletion against a target's ability to find substitutes or bypass sanctions. Success depends heavily on how easily the targeted input can be replaced, not just the severity of the supply cut. The simulation found that if substitutes are readily available, even severe restrictions may fail to stop a war machine. Each layer of enforcement oversight also reduces the effective window for sanctions to work.

Automated cloud deployment pipeline erases emergency fix, causing repeated outage · ShortSingh