What variables best predict U.S. stock market returns? In his June 2016 paper entitled “Which Variables Predict and Forecast Stock Market Returns?”, David McMillan examines the power of 25 variables to predict excess return (relative to the 3-month U.S. Treasury bill yield) of Shiller’s S&P Composite Index both in-sample and out-of-sample. He chooses variables based on connectedness to expected cash flow/dividends and risk and assigns them to five groups:
- Financial ratios: dividend-price, price-to-earnings, cyclically adjusted price-to-earnings (CAPE or P/E10), Tobin’s Q and market capitalization-to-Gross Domestic Product (GDP).
- Economic: GDP cycle, GDP acceleration (rate of change in GDP growth), consumption growth, 10-year to 3-month Treasuries term spread and inflation.
- Labor: wage growth, unemployment, natural rate of unemployment, productivity growth and labor market conditions.
- Housing: house price growth, house affordability, home ownership, housing supply and new house sales.
- Other: University of Michigan Consumer Sentiment, Purchasing Managers Index, National Financial Conditions Index, leverage and non-financial leverage.
He employs regressions to test in-sample predictive power. He then tests out-of-sample forecasts starting in 2000 using various forecast methods and accuracy measures and considering both single-variable and multi-variable models. Using the specified data series as available during 1973 through 2014, he finds that:
- From in-sample regressions spanning the full sample period, the most consistent predictors are: dividend-price ratio, price-to-earnings ratio, CAPE, GDP acceleration, inflation, the natural rate of unemployment and consumer sentiment.
- From 2000-2014 out-of-sample forecasts, models employing multiple predictive variables almost always outperform all those using only one.
In summary, evidence suggests that investors should combine several different kinds of predictors when estimating expected stock market returns.
Cautions regarding findings include:
- Some predictors have revision bias (current historical series do not match as-released data), thereby confounding both in-sample and out-of-sample tests. See, for example, “Productivity and the Stock Market”.
- Shiller’s S&P Composite Index data are “blurry” since monthly values are averages of daily values.
- The sample is very short for analysis of CAPE, consisting of only about four independent 10-year average earnings measurement intervals.
- Testing many variables, methods and performance metrics on the same stock market return data introduces snooping bias, thereby overstating expectations for the best (luckiest) predictors.
- The author does not address economic import of findings via any market timing strategies.
See also:
- “Enhancing Stock Market Prediction with Distilled Economic Variables”
- “Chicago Fed NFCI as U.S. Stock Market Predictor”
- “KCFSI as a Stock Market Return Predictor”
Frequently Asked Questions
What is an excess stock market return and how is it used in return prediction studies?
Excess stock market return is the return on a stock index in excess of a risk-free rate, here the 3-month U.S. Treasury bill yield. In David McMillan's June 2016 study, the excess return of Shiller's S&P Composite Index is the quantity that 25 candidate variables (financial ratios, economic, labor, housing and other measures) are tested to predict, using data from 1973 through 2014.
Which variables best predict U.S. stock market returns?
Stock market return predictors that were most consistent in full-sample in-sample regressions were the dividend-price ratio, price-to-earnings ratio, CAPE, GDP acceleration, inflation, the natural rate of unemployment and consumer sentiment. In 2000-2014 out-of-sample forecasts, models using multiple predictive variables almost always outperformed models using only one. The evidence suggests investors should combine several different kinds of predictors when estimating expected stock market returns.
How are stock market return predictors tested in-sample and out-of-sample?
Stock market return predictors in this study were tested with regressions for in-sample predictive power over 1973-2014. Out-of-sample forecasts then started in 2000 and used various forecast methods and accuracy measures, comparing single-variable and multi-variable models. The 25 variables were chosen for their connection to expected cash flow/dividends and risk and grouped into financial ratios, economic, labor, housing and other categories.
Are multi-variable models better than single-variable models for forecasting stock returns?
Multi-variable models almost always outperformed single-variable models in 2000-2014 out-of-sample S&P Composite Index excess return forecasts. This holds even though individual valuation measures like the dividend-price ratio, price-to-earnings ratio and CAPE were among the most consistent single in-sample predictors. The finding supports combining different kinds of predictors, though the study does not test the economic value of any resulting market timing strategy.
What are the limitations of using economic and valuation variables to predict stock market returns?
Stock market return prediction tests like this one are limited by revision bias, since current historical series for some predictors do not match as-released data. Shiller's S&P Composite Index data are also "blurry" because monthly values are averages of daily values, and the sample covers only about four independent 10-year earnings intervals for CAPE. Testing many variables, methods and metrics on the same data introduces snooping bias that overstates expectations for the luckiest predictors, and the study does not assess market timing profitability.