Open-Source Severity Scorer and pytest CI Gate for Red-Team Findings in Banking LLMs

The article argues that binary red-team reporting for LLM assistants fails to guide release decisions because it ignores differences in harm severity. The author describes an Apache-2.0 open-source Python scorer that converts red-team findings into a typed data model with severity levels and harm categories. The scorer is designed for UAE banks but applies to any release that must pass a risk gate. It can run as a pytest check in CI to block releases that fail defined criteria. The author notes that the tool includes no numeric weights by default, so it reports when a score cannot be computed rather than inventing one.