Why LLM Red-Team Reports Need Reproducibility, Not Just Screenshots

A software engineer argues that most AI red-teaming reports are little more than screenshots of bad model outputs, which cannot be independently verified or rerun. Credible reports, the author contends, must include a fixed and versioned probe corpus, raw prompts and model replies, and a SHA-256 checksum so compliance reviewers can verify nothing changed. A meaningful 0–100 score derived from how many probes a model failed should replace vague qualitative assessments, enabling CI pipelines to block deployments that exceed a set risk threshold. The author has built a small toolkit covering 35 probes across 17 attack classes, with a free 8-probe scan available without requiring an account. The core argument is that reproducibility transforms red-teaming from an anecdotal claim into a verifiable, auditable record.