Objective research and reviews to aid investing decisions
 Looking for information on debt consolidation loans? Want to know more about reverse mortgages? |
The zero-sum S&P 500 futures market involves three categories of players: commercial hedgers; non-commercial traders (large speculators); and, non-reportable traders (small or retail speculators) representative of the public. The Commodity Futures Trading Commission collects and publishes their aggregate positions (short, long and spread) for each asset in a weekly Commitment of Traders report. Are the behaviors of these groups in trading S&P 500 index futures reliable indicators of future stock market direction? If so, is this predictive power stable over time? Using the Historical Commitments of Traders Reports Futures and Options Combined available from CFTC and corresponding S&P 500 index data from late March 1995 (the earliest available for futures) through March 2008 (a total of 675 weeks), we find that:
The following graph depicts the S&P 500 index futures short-long ratio for the three categories of traders over the entire sample period. It shows that:
The short-long ratio for commercial hedgers is approximately balanced (mean short-long ratio 1.03), with low volatility (standard deviation 0.10). This group was net short during the unwinding of the Internet bubble during 2000-2002, preceded by a slightly long bias and followed by a slightly short bias. The commercial traders may be the most conservative and research-oriented (informed) of the three categories.
The large speculators have mostly been net short (mean short-long ratio 1.52) , and their short-long ratio is very volatile (standard deviation 0.70). They were, however, net long during the initial stage of the Internet bubble collapse and for parts of 2005-2007. The large speculators are perhaps the big risk-takers.
The retail speculators tend to take the opposite side of the positions of the commercial traders (mean short-long ratio 0.82 with standard deviation 0.22). Theses traders were notably long during the 2000-2002 market decline, but they were also long during much of the subsequent recovery. This group is perhaps the least likely to have valuable private information.
For more precise tests of the leanings of these three groups, we turn to scatter plots.
The following scatter plots relate the short-long ratios of commercial hedgers and small speculators to the change in the S&P 500 index over the next four weeks for the entire sample period. The relationship for large speculators (not shown) is notably weaker.
The distribution for commercial hedgers leans a little to to the left, suggesting that this category of futures traders tends to get shorter (longer) ahead of 4-week stock market declines (advances). In other words, they tend to time correctly. The Pearson correlation for this distribution is -0.31 and the R-squared statistic is 0.10, indicating that the weekly short-long ratio explains 10% of the movement in the S&P 500 index over the next four weeks.
The distribution for retail speculators leans to the right, suggesting that this category tends to get longer (shorter) ahead of 4-week stock market declines (advances). In other words, they tend to time incorrectly. The Pearson correlation for this distribution is is -0.28 and the R-squared statistic is 0.13, indicating that the weekly short-long ratio explains 13% of the movement in the S&P 500 index over the next four weeks.
Is the predictive power of the behavior of the trader categories stable over time?
The next chart summarizes the relationships between short-long ratios by category of traders and the change in the S&P 500 index over the next four weeks for three different subperiods: 3/95-3/00 (mostly bullish); 4/00-2/03 (mostly bearish); and, 3/03-3/08 (mostly bullish). It shows that:
During 3/95-3/00, commercial hedgers tend to be right, and large and retail speculators tend to be wrong.
During 4/00-2/03, commercial hedgers tend to be right, and retail speculators tend to be wrong. The large speculators are uninformative.
During 3/03-3/08, all three groups are uninformative, suggesting the possibility that the markets for stocks and stock futures markets have adapted to extinguish the informativeness of S&P 500 futures positions.
Are extreme values of the short-long-ratio for commercial hedgers informative?
The final chart recasts the relationship between the short-long ratio of commercial hedgers and the four-week future return on the S&P 500 index by arranging the former series from lowest value to highest value for the 3/03-3/08 subperiod. The horizontal axis is therefore not time-sequential. We set the scales on the vertical axes to make the graphs overlap for ease in seeing how much they do or do not vary together. This format facilitates understanding whether the relationship is stronger or noisier for some levels of the short-long ratio than for others.
The chart generally confirms the lack of a recent relationship between the short-long ratio of commercial hedgers and future stock returns. Even for extreme values of the ratio, the subsequent behavior of stocks is erratic or (for extremely low ratios) perhaps unexpected.
In summary, aggregate S&P 500 index futures positions by trader category, as reported in weekly Commitments of Traders reports, may in recent years have lost any significant power to predict the behavior of the stock market.
For related research, see Blog Synthesis: Sentimental Journey, encompassing a broad range of equity market sentiment measures.
