SShortSingh.
Back to feed

Shadow Traffic Replay Catches AI Code Regressions That Unit Tests Miss

0
·2 views

AI-suggested code changes can pass all unit tests and static reviews yet still fail under real-world request patterns, such as burst traffic or edge-case data states. A more reliable verification method involves deploying the candidate change to a shadow service and replaying a full day of actual production traffic against it. The approach compares status codes and response bodies between a baseline and candidate service for each replayed request, producing a reproducible mismatch list. A sample Python harness is provided that reads a JSONL request log, skips write methods to avoid side effects, and strips sensitive headers before forwarding requests to both services. Because the shadow server is disposable and does not need to be production-grade, the cost of running frequent comparisons remains low.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Seven Architectural Reasons AI Agents Fail in Production — And How to Fix Them

A software engineer who has debugged AI agent deployments for a fintech, a logistics firm, and a SaaS vendor argues that most production failures stem from flawed architecture, not the underlying language model. The core problem, they contend, is that agent frameworks rarely answer two critical questions clearly: what information enters the context, and when should the loop stop. Among the most common failures identified is the absence of a defined terminal state, which can cause agents to declare tasks complete in unintended or harmful ways. A second major vulnerability involves prompt injection, where untrusted content retrieved by tools manipulates the agent into taking unauthorized actions. The author concludes that as AI agents scale, the greater risk is not insufficient intelligence but a lack of explicit design around stopping conditions and trust boundaries.

0
ProgrammingDEV Community ·

NocoBase blocks .doc and .xls uploads due to content-based MIME detection flaw

NocoBase's file manager rejects legacy Word and Excel uploads even when their MIME types are correctly listed in the allowlist, as discovered during testing on version 2.1.23. The platform inspects file contents rather than relying on browser-declared Content-Type or file extensions to determine MIME types. Legacy .doc and .xls files are identified as application/x-cfb due to their shared OLE2 Compound File Binary format, while modern .docx and .xlsx files register as application/zip — neither matching the standard Office MIME types users typically configure. To successfully permit these uploads, administrators must add application/x-cfb and application/zip to the allowlist instead of the conventional MIME type strings. This behavior particularly affects Japanese and Chinese enterprises that still rely heavily on legacy Office file formats.

0
ProgrammingDEV Community ·

Developer Builds Battery-Efficient Offline Geofencing App to Auto-Silence Phones

A developer created Muffle, an Android app that automatically adjusts phone sound settings based on the user's location, after an embarrassing library incident sparked the idea. The app uses Google's Geofencing API with a hybrid location accuracy approach to detect when a user enters or exits defined zones such as libraries, clinics, or mosques. To avoid excessive battery drain, the developer opted for balanced power accuracy mode instead of high-precision GPS, reducing unnecessary radio wake-ups. A loitering delay parameter was added to filter out false triggers caused by brief proximity to a geofenced area. The biggest challenge proved to be OEM-specific battery management on devices like Xiaomi and Samsung, which aggressively killed background services regardless of correct implementation.

0
ProgrammingDEV Community ·

Developer finds two-thirds of architecture rules unenforceable by automated tools

A developer maintaining a plain-English architecture document for a platform they built with an AI coding agent wanted to test how much of it could be automatically verified. After analysing all 24 documented decisions, only 8 turned out to be checkable with deterministic tooling, while 8 were unverifiable process rules and the rest were contextual knowledge statements. The developer built a custom checker that caught 13 real violations on its first run, including frontend components bypassing the service layer for months undetected. Three invariants failed to translate into checks for distinct reasons: missing structural boundaries in the code, the need for cross-file comparison logic, and semantic understanding beyond pattern matching. The exercise revealed that architecture documents often describe systems that do not yet structurally exist in the codebase, making enforcement impossible until the code catches up.