Tools for Defeating Data Snooping

Suppose ten stock market timing strategies out of 10,000 beat the market for ten years running. Are they true outperformers, or just lucky? Multiple hypothesis testing methods address that question by controlling for luck. What are these methods, and how should researchers use them? In their November 2019 paper entitled “An Evaluation of Alternative Multiple Testing Methods for Finance Applications”, Campbell Harvey, Yan Liu and Alessio Saretto:

1. Address the scope of the multiple testing problem in empirical financial economics.
2. Summarize multiple testing methods based on conventional (frequentist) hypothesis testing.
3. Simulate performance of different methods across a variety of testing environments.

Their goal is to provide a menu of choices to help researchers improve inference in financial economics. Based on theory and simulations, they conclude that: (more…)