Developer Documents Three Self-Inflicted Errors While Backtesting a Trend-Following Strategy
A quantitative developer building a trend-following strategy on daily futures bars documented a full day of work, including three separate mistakes in their own verification process. Expanding the backtest from three to nine Russian futures families actually improved performance, raising the Sharpe ratio from 0.56 to 0.83 and cutting maximum drawdown from 19% to 12%, with out-of-sample data outperforming in-sample results. A check that flagged the strategy for concentrating 49% of returns in five days was found to be asymmetric, as applying the same cut to the buy-and-hold benchmark revealed the concentration was a market-wide property, not a flaw in the rule. A separate cost-stress test produced a suspiciously better result, which turned out to be a bug where a hardcoded label masked the wrong set of parameters being computed. After correcting all three errors, the strategy showed robust performance across windows of 30 to 200 days, with costs needing to be inflated roughly fifty-fold before the rule lost its edge over simply holding long.
