Sports betting model's 3,671 predictions failed to beat the bookmaker

A developer logged every probability their sports-betting model produced before matches over 16 days, yielding 3,671 settled predictions. On standard metrics the model appeared strong, with a Brier score of 0.1629 where lower is better. However, when compared against bookmaker probabilities, with the bookmaker's margin removed, on 510 matching predictions, the model did not come out ahead. The author argues that any model comparison needs a clear baseline, a count of paired cases, and confidence intervals. They have published a public repository so readers can reproduce the figures and apply the same evaluation method to other systems, such as classifiers and LLM pipelines.