Lucky Test Portfolio Construction Decisions?

Do test portfolio construction decisions in published research on stock return predictors impound bias by fitting the noise (capturing luck) in historical returns? In his November 2022 paper entitled “Looking Under the Hood of Data-Mining”, Mathias Hasler re-evaluates research published in academic journals on 92 stock return predictors by testing alternatives for 12 portfolio construction decisions (for example, rebalance annually or monthly). He focuses on how original construction decisions correlate with noise in historical returns. Specifically, he compares returns of predictor test portfolios as specified in the original research to those constructed with a random combination of research decisions, both in-sample (original research sample period) and out of sample. He postulates that:

• An in-sample return difference may reflect the correlation of test portfolio construction decisions with either a predictable or an unpredictable (noise) part of historical returns.
• An out-of-sample return difference, however, reflects only the correlation of decisions with the predictable part of historical returns.
• Thus, the difference between in-sample and out-of-sample return differences estimates statistical biases in original portfolio construction decisions. This estimate is an upper bound of bias only, because investors exploiting published research may suppress out-of-sample return predictability.

He therefore conducts further tests to assess the roll of investor out-of-sample exploitation of return predictors. Using data to replicate the 92 stock return predictors both in-sample and out-of-sample during 1926 through 2021, he finds that: (more…)