Passing 680 tests proved nothing: how a hidden parser bug exposed a testing blind spot

·1 views

A developer deliberately introduced a bug in tomlkit's timezone parsing — flipping negative UTC offsets to positive — and found that all 680 official conformance tests still passed. The tests failed to catch the error because the expected values were generated by the very same function being tested, meaning the parser and its expectations drifted together invisibly. The flaw only affected datetime cases; integer and string tests, which used independent expected values, caught similar breaks within seconds. To address this systemic issue, the developer packaged a practice called 'falsifiable-tests' as an AI agent skill, requiring that every test be deliberately broken before it is trusted. The same method was applied to three open-source projects, with findings reported upstream to tomlkit, tenacity, and click.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Your App's Audit Log Must Complement, Not Copy, Your Auth Provider's History

Identity providers record authentication events such as factor challenges and session data, but they cannot capture internal application decisions like device-risk scoring, recovery eligibility, or agent overrides. For SOC 2 compliance, organizations need their own append-only audit logs that document what each authentication event meant within their specific control environment. The two records should be linked using stable correlation identifiers rather than duplicating provider payloads into a second database. OWASP guidance supports collecting authentication signals but does not treat third-party event feeds as a complete record of private application logic. The practical distinction is clear: provider history confirms that an authentication event occurred, while the application audit log explains what the system decided to do about it.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Laptop Sleep Cycle Exposed a Critical Python Timeout Clock Bug

A software developer spent 48 hours debugging a worker process that appeared idle while its job deadline silently expired. The root cause was the use of Python's time.time() function to calculate timeout budgets, which relies on wall-clock time that can jump forward or backward — for example, when a laptop wakes from sleep. This caused one job to return immediately after a sleep event and another to run far beyond its intended time limit, while unit tests passed normally since they never triggered clock changes. A log entry showing a remaining budget of -1842.7 seconds was an early clue, but was initially dismissed as a formatting error rather than a clock anomaly. The fix involves replacing time.time() with time.monotonic(), which uses a forward-only clock suitable for measuring elapsed durations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Dev Series: How Thumb Penetration in VRM Animations Is Corrected and Smoothed

A developer working on a VRM animation pipeline from live-action video has published the fifth article in a technical series, focusing on correcting thumb-finger intersection issues. When the thumb penetrates the index finger during animation, simply pushing it away distorts the original gesture, so the correct fix repositions the thumb on top of the index finger at a defined 18mm clearance threshold. The correction amount, called a "gate," is smoothed over time using rank filters and a Gaussian blur to eliminate brief glitches and unnatural step-like movements. Five failed smoothing approaches were documented, including binary thresholding and applying the gate to the wrong pose, before a working solution was found. The series continues with a forthcoming article on fixing a skinning issue that causes the wrist to crease.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Five Hidden Bugs Let a 1:1 Contrast Logo Pass Every Automated Accessibility Check

A technical post-mortem on DEV Community revealed five critical defects in a PNG validation pipeline that allowed a logo with 1.00:1 contrast ratio — essentially invisible — to clear all automated checks. Memory mismanagement was a key flaw: the decompression logic consumed up to 40 MB per request instead of the claimed 18 MB, risking out-of-memory crashes under concurrent load. A declared queue size constant was never actually instantiated, meaning the system had no real back-pressure mechanism and could be overwhelmed during traffic bursts. Additional bugs included a grayscale bytes-per-pixel miscalculation that caused silent data corruption and a synchronous decompression step that could stall the entire event loop. The author published a hardened rewrite enforcing bounded queues, streaming decompression, and correct pixel-format handling to prevent these failures in production.

0 comments Read more at DEV Community