How AI Agents Should Handle Test Results After a Dependency Version Upgrade

When a software dependency upgrades to a new version, a previous PASS result from an AI agent remains valid only for the version it was tested against, not the newer one. The article argues that deleting old results loses useful history, while silently applying them to new versions misrepresents what was actually proven. Instead, the recommended approach is to preserve historical observations with their original scope and mark the new version's status as 'not established' until a fresh check is run. A new observation should be created for each distinct version, environment, or configuration being tested, keeping outcomes tightly bound to their specific context. If two runs produce conflicting results under apparently identical conditions, the disagreement should be recorded and investigated rather than resolved by defaulting to the most recent result.