Python library 'honest-eval' tackles two common ways backtests produce false results
A developer has released honest-eval, a dependency-free Python library designed to eliminate two major sources of misleading backtest results in trading strategies and predictive models. The first issue is data leakage from random train-test splits on time-series data, where future samples contaminate training sets and inflate performance metrics; the library's temporal_split function enforces a strict chronological split with an optional embargo period. The second problem is selecting strategy variants purely by mean performance, which rewards noise over genuine edge when sample sizes are small; the select_best_variant function addresses this by pairing variants against a baseline and requiring the lower confidence bound to exceed zero. An optional recency-weighting feature using Kish's effective sample size prevents a few heavily weighted samples from distorting variant selection over time. The library is available via pip and the source code, along with mutation-verified tests, is published on GitHub.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in