Blog - Investing Notes

January 19, 2007 - Some Notes on Variability of Stock Market Returns (Updated 1/20/07)

How should the variability of stock market returns shape the outlooks of short-term traders and long-term investors? How strong is the tailwind of the general drift upward in stock prices? How powerful is the turbulence of variability? Does the tailwind ever overpower the turbulence? Using weekly adjusted closing data for the S&P 500 index during 1950-2006 (57 years), we find that...

The following chart presents three statistics for random S&P 500 index entry points with varying holding periods from four weeks to 208 weeks (four years) during 1950-2006. The chart shows that:

(1) The likelihood of having a gain in the index ranges from about 60% for the average four-week holding period to around 85% for average holding periods of three to four years. (For the average one-week holding period, the probability of gain is about 57%. The probability of gain for shorter and shorter holding periods trends down toward 50%.)

(2) The mean return ranges from less than 1% for four-week holding periods to nearly 40% for four-year (208-week) holding periods.

(3) The standard deviation of returns is mostly higher, but over the long haul increases more slowly, than the mean return. The mean crosses above the standard deviation as holding periods approach four years. Said differently, turbulence swamps the tailwind for short holding periods, but not for long holding periods.

The next chart focuses on the mean return for random S&P 500 index entry points with varying holding periods during 1950-2006, with error bars one standard deviation (one sigma) above and below the mean. The chart shows that:

(1) For short holding periods, one sigma confidence intervals have a significant proportion of losses.

(2) For long holding periods, one sigma confidence intervals have a very small proportion of losses. In fact, one sigma confidence intervals for holding periods 188 weeks and longer have no losses. For higher confidence levels, however, no-loss holding periods would be longer.

If returns were normally distributed, these error ranges would represent 68% confidence intervals (returns for 68% of a large sample of randomly selected entry points for each holding period would fall within the error ranges). However, these returns are likely not normally distributed (probably more fat-tailed and skewed), so a one-sigma confidence interval equates to a confidence somewhat different from 68%. The "Probability of Gain" graph above indicates a higher level than 68%.

The final chart displays two additional statistics for random S&P 500 index entry points with varying holding periods during 1950-2006. The chart shows that:

(1) The ratio of the mean return to the standard deviation (return per unit of risk, something like the Sharpe ratio), increases steadily from below 0.20 (turbulence >> tailwind) for four-week holding periods to over 1.00 (turbulence ~ tailwind) for four-year holding periods. (For the average one-week holding period, the ratio is about 0.09.)

(2) However, the number of independent observations across the entire sample falls quickly (initially much faster than the ratio of mean to standard deviation rises) as the holding period lengthens. For four-year holding periods, the number of independent observations is only 14, indicating low reliability for out-of-sample inference. This decline in reliability undermines the apparent comfort of long holding periods.

Note that this 57-year sample contains a fairly small number of business cycles (about ten expansions and recessions). From the perspective that the upward drift in aggregate stock prices is closely related to business cycle characteristics, the sample is very small and the statistics of low reliability. Frustratingly, changes in the financial environment (regulations, risk management techniques, technology) probably diminish the predictive power of old business cycles. In general, the evolution of economic (and other social) systems clashes with the statistical appetite for large samples.

In summary, the above results suggest that:

Long-term investors:

- Should probably check account balances only infrequently (see our blog entry of 5/19/06); and,

- Should perhaps consider large deviations (comparable to the long-term standard deviation) above (below) the long-term trend as times to decrease (increase) equity exposure.

Short-term traders:

- Must trade frequently to achieve statistically reliable results, compounding relatively small gains rapidly to avoid falling behind the long-term upward drift of the broad market;

- Must have techniques that reliably beat the drift to ensure that they can consistently cover trading costs; and,

- Should expect a significant proportion of losing trades and consequent capital drawdowns.

The market seems intrinsically resistive to reliable inference, but stock market forecasters should probably forecast infrequently (at long horizons) and always bullishly to maximize their accuracy rates.

For related research, see Blog Synthesis: Volatility Effects.

Reader Herb Geissler of Pittsburgh PA, who is active in the local chapter of the American Association of Individual Investors (leading the Rational Investing Special Interest Group), offered the following comments on this entry:

"You miss a key point of short-term trading -- avoid losses during periods of market weakness by resting in cash or proactively going short. Experienced position traders (holding for weeks or months, not swing trading in days or weeks) have developed indicators that avoid poor market conditions. I am in cash about one third of the time based on MACD, ADX and McClellan Oscillator, among other indicators. I routinely average more than 16-24% annual gains by not letting the market take back my hard won gains.

"There are dozens of methods for profiting in the market (and hundreds for losing). The discipline of mastering a tested and proven method, and adhering to that method, separates the winners from the losers. I am pleased if more than half of my trades are profitable, with the average gain more than triple the average loss. The percentage of winning trades depends primarily on sound stock selection, while the ratio of average gain to average loss depends mostly on good entry/exit discipline. I meet these objectives more easily when markets exhibit stronger trends than they have over the past three years."