Which economic and market variables are most effective in predicting U.S. stock market returns? In his October 2018 paper entitled “Forecasting US Stock Returns”, David McMillan tests 10-year rolling and recursive (inception-to-date) one-quarter-ahead forecasts of S&P 500 Index capital gains and total returns using 18 economic and market variables, as follows: dividend-price ratio; price-earnings ratio; cyclically adjusted price-earnings ratio; payout ratio; Fed model; size premium; value premium; momentum premium; quarterly change in GDP, consumption, investment and CPI; 10-year Treasury note yield minus 3-month Treasury bill yield (term structure); Tobin’s q-ratio; purchasing managers index (PMI); equity allocation; federal government consumption and investment; and, a short moving average. He tests individual variables, four multivariate combinations and and six equal-weighted combinations of individual variable forecasts. He employs both conventional linear statistics and non-linear economic measures of accuracy based on sign and magnitude of forecast errors. He uses the historical mean return as a forecast benchmark. Using quarterly S&P 500 Index returns and data for the above-listed variables during January 1960 through February 2017, he finds that:
- Full-period in-sample tests indicate that Fed model, value premium, PMI and equity allocation are significant predictors of S&P 500 Index returns (plus dividend-price ratio and q-ratio for total returns only).
- For out-of-sample tests based on mean squared error, (whether rolling window or inception-to-date) only PMI beats the historical mean. However, several variables, multivariate combinations and equal-weighted combinations have lower average forecast errors and lower variance forecast errors than the benchmark.
- For out-of-sample tests based on sign of forecast:
- Among rolling window forecasts, eight of 18 individual variables beat the 60% success rate of the historical mean, with PMI highest at 66%. All multivariate and equal-weighted combinations match or beat the benchmark.
- Among inception-to-date forecasts, only two of 18 individual variables and two combinations beat the 67% success rate of the historical mean.
- For out-of-sample tests based on Sharpe ratio of return forecasts:
- Among rolling window forecasts, 14 of 18 individual variables beat the historical mean, as do two of four multivariate combinations and all equal-weighted combinations. Ten of 28 variables/combinations have Sharpe ratios more than double that of the historical mean.
- Among inception-to-date forecasts, only four variables/combinations (same as those for sign of forecast) beat the historical mean, most notably term structure and PMI.
- Poor predictive performance across variables comes from large unsystematic errors, not steady underperformance of the historical mean.
- Forecasts are generally more accurate during bear markets and economic contractions than during good times.
- Short-selling restrictions generally improve forecast accuracies.
- Overall, results suggest that:
- The term structure offers the best set of out-of-sample forecasts, with PMI in second place.
- Based on Sharpe ratio of return forecasts, the most important for investors, the rolling window term structure is optimal.
- Results are very similar for S&P 500 Index capital gains and total returns.
In summary, evidence indicates that U.S. stock market forecasts based on the term structure of interest rates (first) and PMI (second) are consistently superior to those based on historical mean return.
Cautions regarding findings include:
- Forecasted Sharpe ratios are gross, not net. Accounting for trading frictions and shorting costs, which vary considerably over the sample period, would reduce Sharpe ratios.
- Testing many variables, combinations of variables, prediction methods and prediction success measures on the same dataset introduces considerable data snooping bias, such that the best-performing predictors overstate accuracy.
For other perspectives, see:
Frequently Asked Questions
What is the term structure of interest rates and how is it used as a stock market predictor?
The term structure of interest rates, measured in the study as the 10-year Treasury note yield minus the 3-month Treasury bill yield, is a variable used to forecast U.S. stock returns. In David McMillan's 2018 study, it was one of 18 economic and market variables tested for predicting one-quarter-ahead S&P 500 Index returns over January 1960 through February 2017.
Does the term structure of interest rates predict S&P 500 returns better than the historical average?
Yes, the term structure offered the best set of out-of-sample S&P 500 forecasts among 18 variables tested, with the purchasing managers index (PMI) in second place. Based on Sharpe ratio of return forecasts, the rolling window term structure was optimal, and results were very similar for capital gains and total returns. Only PMI beat the historical mean on out-of-sample mean squared error, so the advantage depends on the accuracy measure used.
How are stock market return predictors tested using rolling and recursive out-of-sample forecasts?
The study tests one-quarter-ahead forecasts of S&P 500 capital gains and total returns using both 10-year rolling windows and recursive (inception-to-date) estimation, with quarterly data from January 1960 through February 2017. Forecasts from individual variables, four multivariate combinations and six equal-weighted combinations are compared with a historical mean benchmark. Accuracy is judged by linear measures such as mean squared error and by non-linear economic measures based on sign of forecast and Sharpe ratio.
How does PMI compare with valuation ratios like the dividend-price ratio or CAPE for forecasting stock returns?
PMI outperformed valuation-based variables in out-of-sample testing in this study, being the only one of 18 variables to beat the historical mean on mean squared error and posting the highest rolling-window sign success rate at 66% versus 60% for the benchmark. Dividend-price ratio and q-ratio were significant in-sample only for total returns, while PMI was significant in-sample for both return types. Term structure still ranked first overall, with PMI second.
What are the limitations of using economic variables to forecast stock market returns?
The study's forecasted Sharpe ratios are gross, not net, so trading frictions and shorting costs would reduce them. Testing many variables, combinations, prediction methods and accuracy measures on the same dataset introduces considerable data snooping bias, so the best-performing predictors likely overstate accuracy. Poor predictive performance across variables also stemmed from large unsystematic errors, and forecasts were generally more accurate in bear markets and economic contractions than in good times.