SShortSingh.
Back to feed

How one shared staging environment cost nine teams years of wasted debugging

0
·1 views

A software engineering team at an unnamed company spent years sharing a single staging environment among nine teams, leading to chronic release delays and misattributed failures. A post-mortem review of eight weeks of staging incidents revealed that 31 of 74 failures were caused by another team's changes, yet were investigated by the wrong team each time. The team solved the problem by building per-pull-request environments using their existing infrastructure-as-code setup, each taking six minutes to spin up and costing under £300 a month in total. A small number of genuinely non-duplicable dependencies, such as a partner test account and a mainframe link, were placed behind an automated pipeline lock instead of a manual booking process. The key takeaway, as noted by engineer Sergey Shinder, was that years of effort had been spent speeding up automation when the real bottleneck was always the shared physical resource that no one had thought to measure.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

New C Code Scoring Tool Flags Risky Functions Using Nesting and Pointer Depth

A lightweight open-source tool called c-code-score assigns a single risk score to C functions based on three factors: nesting depth, pointer depth, and dereference chain length. The score is calculated by multiplying these three metrics, aiming to highlight functions most prone to memory bugs and null-pointer errors. When tested against real-world projects like libXt and libtiff, the score correlated with maintenance churn at rates of 0.52 and 0.38 respectively, outperforming both raw line count and cyclomatic complexity. The tool also works as a feedback loop for LLM-generated C code, allowing developers to iteratively prompt rewrites until scores improve. It is available via pip and requires no external dependencies or complex parsing infrastructure.

0
ProgrammingDEV Community ·

Developer builds rule-based crypto wallet classifier to test on-chain label readability

A developer created a browser-based card game called LabelMe to explore how much of a crypto wallet's classification can be inferred from raw on-chain numbers alone. The project uses ten real Ethereum wallets, each displaying four Nansen-computed metrics — PnL, top trades, balance profile, and counterparty mix — and asks players to assign one of five labels: Smart Money, Exchange, Whale, Contract/Pool, or Regular. Answer keys are derived directly from Nansen API calls rather than the developer's own judgment, making the ground truth as objective as possible. Alongside the game, the developer wrote a minimal threshold-based rule reader — no machine learning model involved — to determine how accurately simple numeric rules can replicate Nansen's labeling. The experiment found that activity volume and token diversity were stronger signals for Smart Money classification than win rate or realized PnL.

0
ProgrammingDEV Community ·

Modular Architecture Offers a Simpler Path to Scaling Large Next.js Apps

As Next.js applications grow, codebases often become difficult to maintain, with features scattered across hundreds of files and multiple teams conflicting over the same code. Common scaling solutions like Multi-Zones and microfrontends address the problem by splitting the application into separate deployable units, but introduce significant operational overhead and do not fix the underlying code organisation issue. Modular architecture, familiar from frameworks like Nuxt and NestJS, instead splits code by feature within a single app, keeping all logic for a given feature in one self-contained folder. A developer has built an open-source package called next-modular to bring this pattern natively to Next.js, allowing each feature module to plug into the app through a single configuration entry. The approach aims to improve maintainability, debuggability, and code reuse without the complexity of running and coordinating multiple independent applications.

0
ProgrammingDEV Community ·

How to Retrofit Spring Security Into a Legacy Internal App When VPN Access Ends

A software developer has published a detailed guide on adding Spring Security to an existing Java Spring Boot application that previously relied solely on VPN-based network access for protection. The article addresses a common real-world scenario where internal business tools — such as accounting systems and admin panels — must handle authentication and authorization as remote and external access needs grow. Using a demonstration financial accounting system built on Java 21 and Spring Boot 3.5, the author walks through retrofitting security onto a live, publicly accessible app with a fabricated but structurally realistic dataset. The guide deliberately recreates actual bugs encountered during a parallel work project, allowing readers to trace each flaw from cause to fix using published code on GitHub. It is aimed at beginner and mid-level developers who have basic Spring Security experience but have not yet tackled row-level or role-based data authorization.

How one shared staging environment cost nine teams years of wasted debugging · ShortSingh