SShortSingh.
Back to feed

How One Team Rebuilt Service Reliability Around Real User Actions, Not Metrics

0
·1 views

A software engineering team overhauled their approach to site reliability after a payment-api incident exposed a critical blind spot: healthy infrastructure metrics had masked an 18-second delay on fraud provider calls, blocking actual customer payments. The team adopted SRE practices not as a job title but as a working discipline, anchoring monitoring around concrete user actions such as submitting a payment or running a nightly bank settlement. They defined success for payment requests as a 2xx HTTP response within five seconds, a threshold derived from 30 days of real request-timing data and correlated support ticket patterns. Each measurable indicator — termed a service level indicator — was tied directly to user-facing outcomes rather than internal system health signals like pod status or CPU usage. The shift was driven by the April incident, where every infrastructure check showed green while customers were effectively unable to complete transactions.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

GetBirthChart Astrology Engine Now Available as Python Package on PyPI

The calculation engine behind GetBirthChart, an astrology platform, has been packaged and published to PyPI as gbc-astro, installable via pip. Previously, using the open-source engine required cloning the repository and configuring the environment manually. The package wraps the same underlying engine — powered by Swiss Ephemeris — without introducing separate astrology logic, covering planetary positions, house systems, aspects, and retrograde states. A key design feature is its handling of unknown birth times: rather than substituting a default value, the library omits time-dependent outputs like the Ascendant and houses. Optional API dependencies for FastAPI integration are available separately, keeping the core package lightweight.

0
ProgrammingDEV Community ·

Solana Cuts Account Rent by 90% via SIMD-0437, Raising Value of Old Deposits

Solana's SIMD-0437 proposal, shipping with Agave 4.2, reduces the lamports-per-byte storage rate from 6,960 to 696 across five phased reductions, cutting the cost to open a standard token account from roughly 0.002 SOL to 0.0002 SOL. The change began rolling out on mainnet in late August 2026 and significantly lowers capital requirements for bulk account creation such as airdrops, which previously locked around 20 SOL for 10,000 recipients. Because the update is a relaxation rather than a forced adjustment, all accounts created before activation still return their original higher deposit when closed, making pre-activation accounts worth ten times more per close than new ones. This creates a finite, non-replenishing inventory of higher-value reclaimable SOL sitting in millions of empty token accounts, failed program deploy buffers, and leftover wrapped SOL accounts. Users and developers stand to recover meaningful SOL by closing these legacy accounts before they are forgotten entirely.

0
ProgrammingDEV Community ·

How Agent State, Memory, and Checkpointing Overlap in AI System Design

In AI agent development, state, memory, and checkpointing are distinct concepts that frequently overlap in practice, making their boundaries difficult to define cleanly. State tracks an agent's current execution context, while checkpointing persists that state so a process can resume after interruption. This persisted state functionally provides short-term memory within a conversation thread, but does not automatically create long-term memory. Long-term memory requires a deliberate selection step, where the application identifies which information from an interaction is worth retaining for future use. Systems like LangGraph reflect this architectural separation by using a checkpointer for thread-level state and a separate store for cross-thread persistent information.

0
ProgrammingDEV Community ·

Database Partitioning and Sharding Explained: Scaling Data Across Servers

As applications grow to serve millions of users, a single database server can become a performance bottleneck due to slow queries, high CPU usage, and storage limits. Vertical scaling — upgrading to a more powerful server — has physical and practical limits, making it insufficient for large-scale systems. Database partitioning addresses this by splitting large datasets into smaller chunks called partitions, using strategies such as range, hash, or list-based rules. Hash partitioning uses a hash function to distribute data evenly across partitions, while range partitioning divides data by value intervals such as user ID ranges or date periods. Sharding extends this concept further by distributing partitions across multiple independent database servers, enabling true horizontal scaling for massive datasets.

How One Team Rebuilt Service Reliability Around Real User Actions, Not Metrics · ShortSingh