Blog - Investing Notes

March 8, 2006 - Where Has All the News Gone?

Our blog entry of 3/7/06 summarizes Jing Chen's synthesis of the theories of physical entropy, information processing and evolution to explain investor psychology and financial market behavior. In this entry, we use a few simple assumptions to explore an information-based perspective of the stock market. Using S&P 500 index data from January 1950 to early 2006 and assuming that daily changes in the index level convey all important information newly available, we find the following:

The following chart shows the distribution of daily changes in the S&P 500 index for 1/3/1950 through 3/2/2006, excluding three extreme outlier data. The range of changes for each bar is 0.2%. For example, 0% represents the range -.01% to 0.1%. The average daily change over these 56+ years is 0.03% with standard deviation 0.89%. Rather than fit a continuous mathematical function to this data, we use the discrete empirical results to assign a probability (P) to each daily change in the index. For the outliers, we arbitrarily assign very small probabilities smaller than the smallest among all the other data. Then we use the formula -log(P) to calculate the informational value of each daily change (see Jing Chen's paper for background).

The next chart plots the amount of new information entering the stock market every 63 trading days (about three calendar months) starting in 1957, along with log[S&P 500 index]. We use this interval because much new financial data appears on a quarterly schedule. The horizontal axis has units of 1,008 trading days (about four years), corresponding with the major political cycle in the United States (presidential terms). The information flow looks like, and is related to, a volatility chart. High volatility indicates a large amount of new information moving into the market. We defer interpretation to a slightly different plot construction below and instead next shift to a close-up.

The next chart is a close-up of the preceding chart, showing the amount of new information entering the stock market every 63 trading days (about three calendar months) from the beginning of 2003 to the present, again along with log[S&P 500 index]. Information flow has slowed over this recent period. Also, visual observation suggests strongly that the two lines have negative correlation. When there are surges of new information, the market goes down. When there is little news, the market rises. There is more on this observation below.

The final chart shows a more sophisticated view of information active in the stock market. For this view, instead of ignoring old information as in the plots above, we assume that the value of information begins to decay the day after it becomes known. By analogy (for lack of theory), we assume information value decays in the same way as radioactivity, with a mean lifetime of one calendar year (a half-life of 175 trading days). So, after 175 trading days, financial information retains only one-half of its original value. After 350 trading days, it retains only one-fourth of its original value. The result is a smoothed and somewhat dampened version of the "moving window" truncation approach above.

This last chart has three major peaks. The first, in the mid-1970s coincides with President Nixon's resignation and the first of two waves of historically high inflation. The second coincides with the 1987 stock market crash. The third, in the early 2000s, perhaps represents a catharsis of irrational Internet exuberance. The unusually long build-up to the third peak coincides with Internet exuberance. The notable dip in the early to mid-1990s coincides with an international quiet period after the dissolution of the Soviet Union.

There is no consistent relationship between the amount of active information in the market and the direction of the market. For example, the correlation between active information in the market and log[S&P 500 index] since January 2003 is strongly negative (-0.94). All news is bad news during this period. However, during 1997-2000 (the second Clinton administration), the correlation is strongly positive (+0.94). Then, all news was good news. (There is no pattern for this correlation with respect to the party of the President.) This inconsistency of correlations indicates that the history of information flow says little about the future. Visual observation of the above chart suggests that quiet periods are not bad for the market, but short loud ones are.

Since late 2002, the amount of information active in the market has been declining. The all frenzy all the time approach of the media is irrelevant. Perhaps information value decays more rapidly in a networked world, because more people get more news more quickly and more more cheaply. Cheap information is low-value information. Information known to all is no-value information. What will drive the plot back up? Something unimaginable, or at least extremely surprising.

In summary, a simple information processing view of the stock market offers an interesting explanatory framework, but little predictive power.

For related research, see our blog entries of:

3/7/06 summarizing Jing Chen's paper cited above;

7/21/05 for a related analysis of stock market volatility; and,

6/14/05 finding that specific kinds of macroeconomic news can be good or bad depending on the phase of the economic cycle.