Some analysts fail to account for data snooping bias in their analyses of market timing indicators. This bias amounts to incorporating pure luck into results by testing many different rule variations or parameter settings within rules (or inhaling the “secondary smoke” of other analysts who have already screened a set of rules/parameters). This luck does not persist out-of-sample. Do any market timing rules generate outperformance after correcting for this bias? In their February 2009 paper entitled “Data Snooping and Market-Timing Rule Performance”, Andreas Neuhierl and Bernd Schlusche assess the profitability of a comprehensive set of simple and complex market timing rules based on fundamental indicators and investor sentiment indicators after correcting for data snooping bias. Simple rules derive from a single indicator, and complex rules derive from multiple indicators. Using thousands of simple and complex rules based on data for the S&P 500 to time the daily close of the S&P 500 index over the period 1980-2007, they conclude that:
Over the entire sample period and a 1981-1994 subperiod, they test 6,792 simple market timing rules and 1,976 complex rules. Over a 1995-2007 subperiod, they test 8,280 simple rules (more types of data available) and 1,976 complex ones. Rules include those based on: financial ratios (earnings-to-price ratio, dividend yield and book-to-market ratio), short-term and long-term interest rates and spreads, expected inflation, measures of investor sentiment (credit spread and put-call ratio), implied volatility index, bond-equity yield ratio and dividend payout ratio.
- In isolation, over the entire 1980-2007 sample period, the best-performing simple (complex) rule outperforms a buy-and-hold strategy by an annualized 2.93% (3.18%).
- Over the the full sample period, no simple or complex market timing rule significantly outperforms a buy-and-hold strategy after correcting for data snooping bias.
- Over 1981-1994 and 1995-2007 subperiods, no simple market timing rule significantly outperforms a buy-and-hold strategy after correcting for data snooping bias.
- Over the 1981-1994 subperiod, certain complex market timing rules do outperform a buy-and-hold strategy even after correcting for data snooping bias and accounting for reasonable transaction costs. However, over the 1995-2007 subperiod, no complex rule significantly outperforms a buy-and-hold strategy after correcting for data snooping bias.
In summary, even though S&P 500 index timing rules based on fundamental indicators and investor sentiment indicators might significantly beat a buy-and-hold benchmark when evaluated in isolation, this outperformance generally evaporates after correcting for data snooping bias. In other words, luck is the dominant differentiator of rule performance.
Frequently Asked Questions
What is data snooping bias in market timing?
Data snooping bias in market timing is the incorporation of pure luck into results by testing many different rules or parameter settings, or by relying on rules other analysts have already screened. This luck does not persist out-of-sample, so apparent outperformance can be misleading. Neuhierl and Schlusche's study tested 6,792 to 8,280 simple rules and 1,976 complex rules on the S&P 500 over 1980-2007.
Do market timing rules based on fundamental and sentiment indicators beat buy-and-hold?
Market timing rules based on fundamental and sentiment indicators generally do not significantly beat S&P 500 buy-and-hold after correcting for data snooping bias. Over 1980-2007, the best simple (complex) rule outperformed by an annualized 2.93% (3.18%) in isolation, but no simple or complex rule was significant after correction. The one exception was certain complex rules in the 1981-1994 subperiod, but not in 1995-2007.
How are market timing rules tested for data snooping bias?
Market timing rules are tested for data snooping bias by evaluating the full universe of rules together and adjusting significance for the number of rules tried, rather than judging the best rule in isolation. Neuhierl and Schlusche applied this to thousands of simple (single-indicator) and complex (multi-indicator) rules for timing the daily S&P 500 close over 1980-2007. Indicators included earnings-to-price, dividend yield, book-to-market, interest rates and spreads, expected inflation, credit spread, put-call ratio, implied volatility, bond-equity yield ratio and dividend payout ratio.
Do complex market timing rules work better than simple ones?
Complex market timing rules, which use multiple indicators, did slightly better than simple single-indicator rules, but only in one period. Over 1981-1994, certain complex rules outperformed buy-and-hold even after data snooping correction and reasonable transaction costs, while no simple rule did. Over 1995-2007 and the full 1980-2007 sample, neither simple nor complex rules significantly outperformed.
Why does market timing outperformance often disappear out-of-sample?
Market timing outperformance often disappears because much of it reflects luck from testing many rules or parameter settings, and luck does not persist out-of-sample. In Neuhierl and Schlusche's study, rules that beat S&P 500 buy-and-hold in isolation generally lost significance once data snooping was corrected for. The authors' implication is that luck is the dominant differentiator of rule performance.