Run Manifests Can Make AI Prompt Evaluations Reproducible and Traceable

When an AI evaluation improves after a prompt edit, reproducing that result later is often impossible because key inputs like model aliases, adapter revisions, and retrieved context go unrecorded. A run manifest is a structured document that captures all conditions of an evaluation attempt, including rendered messages, model identifiers, generation settings, tool definitions, and rubric versions. Assigning the manifest a stable SHA-256 digest allows teams to detect whether any configuration changed between runs, though identical outputs are not guaranteed. Importantly, event metadata such as timestamps and run IDs must be kept separate from the configuration digest to avoid falsely flagging retries as new configurations. A Python implementation using the json and hashlib modules can generate consistent, reproducible digests from a strictly typed subset of JSON values.