SShortSingh.
Back to feed

AI Agent Log Exposes How Missing Baselines Corrupt Measurements

0
·25 views

An AI agent's operational log, reviewed on 2026-09-23, revealed that an earlier count of 16 files could not be reconciled with a fresh count of 14 because the original measurement lacked a defined ruleset or date. A separate audit uncovered a delivery bug in which 6 of 10 letters sent on 2026-09-18 failed to reach named recipients, caused by a sending tool that read header addresses but not the actual delivery envelope. A colleague identified the flaw, prompting the agent to add a guard that now cross-checks every header name against the envelope. Three additional baseline errors were also documented that same morning, including a provenance diff that flagged phantom additions due to a line-count mismatch and a file check that returned false negatives by counting only one dialect of a field name. The log argues that any figure reported without an attached method and date is functionally unverifiable, framing measurement discipline as a core reliability requirement.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer swaps autonomous AI agent for deterministic routing to cut costs and boost control

A developer built an agent that autonomously managed a calendar scheduling assistant, but it repeatedly called the model and databases, making a single message cost over a dollar in tokens. Engineering flaws contributed, yet the developer judged the design unsustainable even after fixes. They rebuilt the system with the same user-facing behavior but a deterministic architecture of AI nodes and switches that tag workflow state and route messages. The new design reduced costs and gave the developer more control and better security by limiting the AI's autonomy, following the least-privilege principle. The author argues this principle deserves more attention from developers building AI systems.

0
ProgrammingDEV Community ·

Detecting Input Drift with PSI Before Model Accuracy Degrades

Deployed machine learning models can lose accuracy even when their code and weights stay unchanged, because the data they receive shifts over time. Ground-truth labels often arrive late, so monitoring input and output distributions offers an earlier warning sign. The article recommends using the Population Stability Index (PSI) to compare live data against a fixed, deliberately chosen baseline snapshot. It explains the PSI formula, offers a NumPy implementation, and suggests common threshold conventions that should be calibrated to each system. The author also advises sanity-checking the metric and measuring its natural noise on small samples before setting alerts.

0
ProgrammingDEV Community ·

Developer builds simple tool to flag salon leads left unanswered for 15 minutes

A developer built a small booking tool for a salon to address customers who leave enquiries that go unanswered while the owner is busy. The system has three parts: a booking form that saves each lead, a follow-up message preview generated immediately from the lead's details, and an owner inbox that shows leads newest first. If a lead goes 15 minutes without being marked as attended, the inbox displays a missed lead badge until the owner clicks to clear it. The backend uses Python with FastAPI and SQLite, while the front end is plain HTML and JavaScript without paid APIs. The developer found that the owner mainly needed a clear view of waiting leads rather than extra features, and the next planned step is connecting a real WhatsApp or SMS service to send the follow-up messages.

0
ProgrammingDEV Community ·

OnePassApps Stylesheet Analysis: 403 Custom Properties, Container Queries and Forced Colours

A team member at OnePassApps, which runs app.onepassapps.com, published a byte-level breakdown of the CSS its live site serves. The home page links a 3 KB pre-JavaScript fallback stylesheet and a 242 KB main stylesheet with long-lived caching. The main file defines 403 named custom properties grouped into families such as brand colours, ink scales, control sizes, motion timings and layout containers. Its 14 font-face blocks for Inter Variable are served from the same host as the CSS, with no third-party font requests and no font preloading. The author also notes that the stylesheet contains no prefers-color-scheme rules, so theme selection is handled in application JavaScript, and that it includes accessibility media queries for forced colours, contrast and reduced motion.