Approval Testing Offers a Rigorous Way to Validate AI Prompt Outputs
Approval testing, a method predating snapshot testing, uses two files per test case to validate outputs: a '.received' file generated at runtime and a '.approved' file committed to the repository after human review. When both files match exactly, the test passes; if they differ, a diff tool highlights the change for a human reviewer to assess and approve. The approach is available across multiple languages including Python, Java, .NET, C++, and JavaScript through the ApprovalTests library. Unlike snapshot testing, which implies correctness, approval testing only claims that a named person reviewed and accepted a specific output at a specific time, creating a built-in audit trail via git blame. However, the method has clear limits: it detects changes effectively but cannot catch errors that were present and approved from the start, such as a prompt that has always mishandled certain data formats.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in