Researcher finds his own AI verification scores hide unmeasured blind spots

A developer published a preprint on arXiv this week revealing that his AI refusal-site verifier could be fooled by minimal mutations, with the cheapest surviving forgery requiring only four bytes of change. Community reviewers quickly identified a deeper flaw: the scoring system only measures obligations that already have refusal sites mapped to them, leaving an unknown number of spec obligations entirely outside the denominator. Of 39 normative obligations listed in SPEC.md, none have a formal artifact mapping them to refusal sites, meaning gaps in coverage are unmeasured rather than confirmed small. A parallel discussion on a separate post about AI agent pricing surfaced the same structural problem — published scores reflect only surviving or kept runs, not the full population of attempts. The author acknowledged all corrections, noted a revised paper version is queued, and stated that building a complete obligation-to-site map is now his next priority.