Developer Ditches Custom Weather Model After NOAA's Free Tool Proved Far Superior

A developer spent four months building a custom weather ensemble model to trade on prediction markets, only to find it performed worse than using no model at all, scoring a Brier score of 0.2858 against a baseline of 0.2439. The model's core flaw was overconfidence — its probability estimates were 2.1 to 4.0 times too narrow — and it carried a systematic temperature bias of up to seven degrees Fahrenheit at the gridpoint level. The root cause was that ensemble members from the same model share systematic biases, meaning 164 agreeing forecasts only confirmed internal consistency, not accuracy. NOAA's National Blend of Models (NBM), a free public product, already solves these problems by applying statistical post-processing and delivering calibrated, bias-corrected, station-level probabilistic forecasts. The updated bot now uses NBM as its primary source with a 0.75 weight, relegating the original raw models to a minor sanity-check role.