### Estimating Snooping Bias for a Multi-parameter Strategy

**March 14, 2014** - Big Ideas, Momentum Investing, Volatility Effects

A subscriber flagged an apparently very attractive exchange-traded fund (ETF) momentum-volatility-correlation strategy that, as presented, generates a optimal compound annual growth rate of 45.7% with modest maximum drawdown. The strategy chooses from among the following seven ETFs:

ProShares Ultra S&P500 (SSO)

SPDR EURO STOXX 50 (FEZ)

iShares MSCI Emerging Markets (EEM)

iShares Latin America 40 (ILF)

iShares MSCI Pacific ex-Japan (EPP)

Vanguard Extended Duration Treasuries Index ETF (EDV)

iShares 1-3 Year Treasury Bond (SHY)

The steps in the strategy are, at the end of each month:

- For the first six of the above ETFs, compute log returns over the last three months and standard deviation (volatility) of log returns over the past six months.
- Standardize these the monthly sets of past log returns and volatilities based on their respective means and standard deviations.
- Rank the six ETFs according to the sum of 0.75 times standardized past log return plus 0.25 times past standardized volatility.
- Buy the top-ranked ETF for the next month.
- However, if at the end of any month, the correlation of SSO and EDV monthly log returns over the past four months is greater than 0.75, instead buy SHY for the next month.

The developer describes this strategy as an adaptation of someone else’s strategy and notes that he has tested a number of systems. How material might the implied secondary and primary data snooping bias be in the performance of this system? To investigate, we examine the fragility of performance statistics to variation of each strategy parameter separately. As presented, the author substitutes other ETFs for the two with the shortest histories to extend the test period backward in time. We use only price histories as available for the specified ETFs, limited by EDV with inception January 2008. Using monthly adjusted closing prices for the above seven ETFs and for SPDR S&P 500 (SPY) during January 2008 through February 2014 (74 months), *we find that:* Keep Reading