Curbing Data Snooping

How should researchers applying machine learning to quantitative finance address the field’s data limitations, which exacerbate data snooping bias? In their October 2018 paper entitled “A Backtesting Protocol in the Era of Machine Learning”, Robert Arnott, Campbell Harvey and Harry Markowitz take a step back and re-examine financial markets research methods, with focus on suppressing backtest overfitting of investment strategies. They introduce a research protocol recognizing that self-deception is easy. Their goal is that the protocol offers the best way to match or beat expectations in live trading. Based on logic and their collective experience, they conclude that: (more…)