Why 'Unknown' Is Not Enough: Lessons From Building Smarter AI Checks

A developer building AI agents found that adding a third outcome — 'could not tell' — to binary pass/fail checks revealed hidden failures that would otherwise go unnoticed. A grep command returning zero was misread as confirming a dangerous line was absent, when in fact the search pattern never matched at all. The developer also found that 'not applicable' results carry risk, as they attract no scrutiny and can mask genuine errors when misapplied. A health check labeled 'alive' was later found to bundle three distinct facts, meaning one could silently fail while the others passed. The broader lesson is that observations and explanations must be tracked separately, and any third value must be clearly defined before it is trusted.