Risk Scores Are Ordinal Rankings, Not Probabilities — And Misuse Causes Real Harm

A common but damaging mistake in software development is treating risk or fraud scores as probabilities, when they are actually ordinal rankings indicating relative risk, not percentage likelihood. Developers often build arithmetic operations on these scores — such as dividing by 100 or averaging across sessions — producing confident-looking numbers that are statistically meaningless. The problem is difficult to catch because such systems often behave approximately correctly, failing only at edge cases in ways that go unnoticed. To obtain a true probability, engineers must calibrate scores against actually observed outcomes through continuous feedback and measurement. The author highlights an ongoing challenge in the industry: no effective method, such as automated linting or documentation, has yet reliably prevented this misuse at the code review stage.