Quantifying Snooping Bias in Published Anomalies

Is data snooping bias a material issue for cross-sectional stock return anomalies published in leading journals? In the September 2017 update of their paper entitled “Publication Bias and the Cross-Section of Stock Returns”, Andrew Chen and Tom Zimmermann: (1) develop an estimator for anomaly data snooping bias based on noisiness of associated returns; (2) apply it to replications of 172 anomalies published in 15 highly selective journals; and, (3) compare results to post-publication anomaly returns to distinguish between in-sample bias and out-of-sample market response to publication. If predictability is due to bias, post-publication returns should be (immediately) poor because pre-publication performance is a statistical figment. If predictability is due to true mispricing, post-publication returns should degrade as investors exploit new anomalies. Their baseline tests employ hedge portfolios that are long (short) the equally weighted fifth, or quintile, of stocks with the highest (lowest) expected returns for each anomaly. Results are gross, ignoring the impact of periodic portfolio reformation frictions. Using data as specified in published articles for replication of 172 anomaly hedge portfolios, they find that:

(more…)