SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer finds data-drift detector crying wolf due to floating-point and HLL noise

0
·7 views

A developer building a pipeline drift-detection tool designed a benchmark of 56 seeded faults, which the tool resolved correctly in 55 out of 56 cases. To check for false positives, four negative control scenarios were added where the correct detector response was silence. The null control — a run with no data changes at all — failed, generating three high-severity alerts. Investigation revealed two root causes: HyperLogLog's probabilistic distinct-count estimates varied by up to 30% between identical runs, and DuckDB's parallel aggregation produced non-deterministic floating-point results that made identical values appear different. The fix was to use exact distinct counts and round floats before comparison, underscoring that measurement determinism is a prerequisite for reliable anomaly detection.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer Builds RPG Character Generator App Using Google AI Studio's Gemini Feature

A developer created 'Mythic Forge,' an interactive RPG Character Portrait and Backstory Generator, as a submission for DEV's Google AI Studio education track. The web app was built using a single structured natural language prompt specifying React, TypeScript, and Gemini API integration. Google AI Studio's 'Build apps with Gemini' feature generated a complete frontend project with character classes, dynamic attribute rollers, narrative backstories, and party management tools. The developer highlighted three standout capabilities: Gemini's transparent step-by-step planning, real-time self-healing code that resolved errors automatically, and the ability to go from a text prompt to a functional app in minutes. The project was successfully prototyped within Google AI Studio's interactive IDE preview using TypeScript, React, Tailwind, and Google Gemini SDKs.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

AI-Augmented Parallel SDLC Replaces Sequential Handoffs With Single Discovery Model

A new software delivery model proposes replacing the traditional chain of handoffs — from analyst to designer to developer to QA — with a single joint discovery session attended by all disciplines simultaneously. Each team member takes their own notes through their own lens, eliminating fidelity loss that typically accumulates as requirements pass from role to role. Development, design, and QA then run in parallel from that shared understanding, converging only at a final quality gate. AI agents are layered in to handle production tasks and project memory, freeing human team members to focus solely on engineering judgment and accountability. Teams that have run this model in production report that the roughly 15–20% of requirements that naturally evolve are caught early and cheaply, rather than surfacing as costly change requests during UAT.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds Plan-First Claude Code Plugin to Tame Autonomous Coding Agents

A developer frustrated with AI coding agents making unreviewed file edits before goals were clearly defined has built an open-source Claude Code plugin called Loomwright. The tool enforces a plan-first workflow where a 'Launch Pad' component drafts a feasibility brief and subtask breakdown before any code changes are made to a repository. Workers run in isolated Git worktrees to prevent overlapping edits, and completed work is parked as a pull request rather than silently merged. Auto-merge is opt-in only and gated behind an explicit trust mechanism, keeping human review as the default checkpoint. The plugin is publicly available on GitHub and installable via the Claude plugin marketplace under the identifier 'loomwright@atelier'.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

EU Cyber Resilience Act Makes Security by Design a Legal Obligation by 2027

The EU Cyber Resilience Act (CRA) elevates 'security by design' from an industry buzzword to a binding legal requirement under Annex I, applicable to all products with digital elements from 11 December 2027. Unlike other controls in the regulation, this baseline requirement carries no 'where applicable' exemption, meaning manufacturers cannot opt out regardless of their product's risk class. The European Commission justified the law partly by estimating that data breaches and disruption attacks cost EU businesses at least €75 billion annually. Experts warn that architecture decisions determining compliance must be made years ahead of the deadline, as retrofitting security onto finished hardware is often technically impossible without a full redesign. An earlier obligation under the CRA — vulnerability reporting under Article 14 — takes effect sooner, on 11 September 2026.

0 comments Read more at DEV Community