SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer finds test suite missed 1,029 lines due to tests built only on clean inputs

0
·1 views

A software developer discovered that a security guard script meant to prevent cross-project file access was blind to 1,029 of 6,309 monitored lines, including 824 executable ones, despite three tests staying green for months. The flaw went undetected because all three tests used only clean, violation-free inputs, meaning a broken detector and a working one produced identical results. The core mistake, the developer explains, is building test suites solely from negative cases — files that should return no violations — which cannot distinguish a functioning rule from a completely blind one. The fix requires every detector test to include both a case that must trigger a finding and one that must not, ensuring neither a permissive nor a blind rule can silently pass. The developer also recommends validating tests against synthetic inputs and previous broken versions of the detector to confirm the new tests genuinely discriminate between correct and faulty behaviour.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

GA4 Standard Reports Show Zero Traffic Since Sept 1 Due to Platform Bug

Google Analytics 4 standard reports began displaying zero or significantly reduced traffic data for many users starting September 1, 2026, due to a confirmed platform-wide reporting bug. Google acknowledged the issue in an official Help Center support thread, noting that Active Users and traffic data were not appearing in standard reports. Crucially, GA4 Realtime reports continued to show visitor activity, indicating that data collection itself was unaffected and no actual traffic loss had occurred. Analysts and teams are advised not to make campaign, budget, or tag configuration changes based on the erroneous figures. Google has not yet specified a resolution timeline, and users are encouraged to cross-check data with independent measurement tools until standard reporting is restored.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Rust API Design: How Sealed Traits Prevent Breaking Changes in Public Libraries

Rust's coherence rules prohibit multiple implementations of the same trait for a single type, and many trait-related changes — such as adding blanket implementations or altering method signatures — are considered breaking changes. When two traits define a method with the same name and a type implements both, the compiler cannot resolve which method to call, resulting in a compilation error. Sealed traits offer a practical solution by allowing traits to be used by external crates but not implemented by them, effectively locking down the set of valid implementors. This pattern is achieved not through a built-in language feature but by placing a private supertrait inside a private module, making it inaccessible outside the defining crate. Sealed traits are especially useful for derived or blanket implementations, where only a controlled set of types should be permitted to satisfy a given trait bound.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Finds Model Size Was Never the Issue in Self-Improving AI Agent Experiment

A developer building a self-improving AI agent tested four language models — ranging from 1B to 30B parameters — across 4,150 LLM calls, hoping a larger model would generate better prompt edits. All four models failed to produce a single promotable edit, with each converging on the same narrow region of the prompt rather than exploring new approaches. The most capable model tested, Mistral 24B, showed directional improvement in three of five iterations but never reached statistical significance. The developer concluded the core problem was not model capability but the search strategy, which lacked any mechanism to step back and explore fundamentally different edit types. Failures were concentrated in multi-label classification, ambiguous categorisation, and keyword over-indexing, issues that minor prompt rewording consistently failed to address.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Apify Actors hidden from Store search until developer completes identity verification

A developer published eleven Actors on the Apify platform in late August, only to find they received zero organic users after a full week despite rewriting descriptions and promoting the tools. A test using the Apify Store API revealed that searching by exact Actor slug returned zero results for all eleven tools, while a lesser-known Actor with just four users appeared normally. The cause turned out to be a default Store filter called 'includeUnrunnableActors', which silently excludes Actors from developers who have not completed identity verification. The developer had not completed KYC because the verification prompt is buried under Actors → Insights → Payouts, not in the main Settings area, making it easy to miss. Once identified, the issue was resolved in roughly four hours, but it took a full week just to diagnose the root cause.

0 comments Read more at DEV Community
Developer finds test suite missed 1,029 lines due to tests built only on clean inputs · ShortSingh