Guru Stock Market Forecasting Grades

Can experts, whether self-proclaimed or endorsed by others (publications), provide reliable stock market timing guidance? Do some experts clearly show better intuition about overall market direction than others?

We have accumulated reviews of the public U.S. stock market forecasts of various investing/trading experts for more than two years. With over 4,100 measurements for 50 gurus, including bulls and bears and technicians and fundamentalists, we have critical mass for: (1) assessing the forecasting acumen of the stock market gurus as a group; and, (2) ranking experts according to the accuracy of their past forecasts. This kind of forecasting ability is different from, but may be related to, stock picking expertise.

Note that the overall assessment of the stock market forecasting ability of experts in aggregate is far more reliable, based on sample size and duration, than the evaluations of individuals.

Note also that this study is not a test of whether the outputs of the experts are interesting, stimulating or useful in ways other than predicting the behavior of the overall U.S. stock market.

We update this page and the underlying guru forecast database about weekly.

Snapshot  -  Notes  -  Individual Gurus -  Q&A

SNAPSHOT

We restrict reviews to publicly available material (freely available on the web), putting ourselves in the place of an individual investor trying to locate value in the marketplace, and mindful of concerns about copyright and trade secrets. Sometimes we find public records on the web sites of the experts themselves and sometimes on web sites of other parties (for example, the business media). This approach helps keep a level playing field for reviews, and it allows readers to check easily the context of forecasts and the reasonableness of our judgments.

The following table (last updated 8/9/08) summarizes the results of our reviews of the publicly available forecasts of 50 experts regarding the future direction of the overall U.S. stock market. For each expert, the table shows the total number of measurable forecasts, the numbers of forecasts we judge to be essentially right and essentially wrong, and the accuracy rate. Accuracies range from a low of 12% to a high of 68%. The overall accuracy of the group based on both raw forecast count and on the average of forecaster accuracies (weighting each individual equally) is 48%.

See the Notes below for discussion of the grading process and for cautions on interpreting both the aggregate and the individual expert accuracy rates.

If you believe that past forecasting performance indicates future accuracy (skill, wisdom), you might want to keep tabs on the forecasts of Ken Fisher (and his self-described analysis methodology, "Forecasting (Macro and Micro) and Future Concepts"). If you believe in reversals of fortune, you might follow Bill Fleckenstein. On the third hand, if you expect mean reversion in forecasting (pure luck), you might prefer the "cone of silence". A fourth alternative is to apply Bayesian updating and weight the predictions of experts according to their evolving track records.

In summary, stock market experts as a group do not reliably outguess the market. Some experts, though, may be better than others.

For an unscientifically collected list of reasons why gurus go wrong, see our blog entry of 5/24/07. For additional comic relief, see our blog entries of 8/2/07 and 8/3/07.

NOTES

Please note these points regarding the results in this table:

We selected experts to be graded based on extensive web searches for public archives offering at least marginal stock market forecast sample sizes. (Readers helped identify some commentators.) There may be data availability bias in the aggregation; some types of commentators may be more likely to offer public commentary than others.

Many of the samples for individual experts are small, rendering confidence in the associated accuracies low. The aggregate sample, however, is large.

Assessing market direction forecasts requires judgment (ours) because they sometimes contain ambiguities, equivocations and/or conditional statements. We expect that our judgment errors tend to cancel each other, but we may have biases. Our detailed judgments are available for inspection via the links at the bottom of this page. Further, each individual review provides a link to source commentaries and articles, so that readers may decide whether the essential forecast language properly reflects commentary context.

When an expert commentary is too vague to assess or does not include a stock market forecast, we either do not include it or (if it is of background interest) we include but do not label it either "right" or "wrong." For experts with small samples, we make a few exceptions for commentaries that include no market direction forecast but do offer some other significant and reasonably testable market-related prediction or recommendation.

When an expert issues weekly (monthly) commentaries, we tend to focus on the behavior of the market in the next week (month), unless the forecast specifies some other timeframe.

When judging whether a forecast is correct, we keep in mind empirical benchmarks based on weekly S&P 500 index data for 1950-2006, as listed below. For example, if a guru says investors should be bullish over the next six months (26 weeks), and the market is up by only 2% over that interval, we would judge the call incorrect. Conversely, if a guru says investors should short the market over the next month, and the market is down only 1%, we would again judge the forecast incorrect (a losing position after trading and carrying costs). In summary, the grading process has normalizing or detrending effects such that the aggregate accuracy should probably be around 50%.

• About 50% of all one-week returns are greater than +0.3%.
• About 50% of all four-week returns are greater than +1.0%.
• About 50% of all 13-week returns are greater than +2.4%.
• About 50% of all 26-week returns are greater than +4.5%.
• About 50% of all 52-week returns are greater than +9.5%.

Assessments of different experts cover different timeframes according to the data available. This fact arguably weakens the case for ranking the experts. An expert who is stuck on bullish (bearish) would tend to outperform in a rising (declining) stock market. This effect should eventually cancel for the entire sample across all experts.

The private (for example, paid subscription) forecasts of experts may be more timely and more accurate than the forecasts offered publicly. As noted above, we restrict reviews to publicly available material to: (1) maintain a level playing field for experts reviewed; and, (2) let readers check the reasonableness of our judgments.

With exceptions as noted, these assessments generally address forecasts of overall market direction, not the performance of stock picks. Most stocks, however, exhibit substantial co-movement with the overall market.

For further reading on the performance of experts in general, and what might distinguish a good market forecaster from a bad one, see Blog Synthesis: The Wisdom of Analysts, Experts and Gurus. Consider also also research on how individuals recollect and process advice as summarized in some of the items at especially the third, sixth and seventh items at Blog Synthesis: Individual Investing.

INDIVIDUAL GURUS

The following names link to detailed performance evaluations. Most of these gurus are included in the snapshot above, but some do not fit the methodology and are therefore not comparable.

Some of these individuals have contested our evaluations of their commentaries. In such cases, individual evaluations note the disputes and provide links to details.

Q&A

A financial advisor with a large investment services company sent a series of questions/comments regarding Guru Grades. In case others have the same questions, here are the queries and responses...

Question: I'm guessing that you get paid by the gurus you hype, no?

No. Neither CXO Advisory Group LLC, nor any of its members personally, receive any payments or other compensation from the gurus reviewed on the site. The reviews are attempts to measure the ability of experts to forecast stock market behavior, not "hype."

Question: I'm flabbergasted by your top "gurus", as I am familiar with them as well, and they are very poor performers. How can you tout a marketing company (Fisher) as a top money management company? Do you research these firms at all?????

The site does not "tout" any companies. It presents reviews of research, some models and some original analyses. The reviews and analyses presented are exactly as described on the site. It is up to readers to decide whether and how this information translates into action for them, or merely flabbergasts them.

Question: Why do you compare what people say in articles/periodicals? It means nothing for professionals/investors! Their actual management/performance would certainly be more beneficial to "private investors and financial advisors."

As stated above, the material cited addresses the following questions: "Can experts, whether self-proclaimed or endorsed by others (publications), provide reliable stock market timing guidance? Do some experts clearly show better intuition about overall market direction than others?" A reader suggested the concept of comparing multiple experts in this way some time ago.

The discussion there further notes that: "This kind of forecasting ability is different from, but may be related to, stock picking expertise."

Results are not conclusive that any individual guru has special ability to forecast market behavior. The distribution of the accuracies of individual gurus about the mean is possibly normal.

If the reviews of guru stock market forecasting records mean nothing to professionals/investors, then professionals/investors will pay no attention to them.

Note that Blog Synthesis: The Wisdom of Analysts, Experts and Gurus catalogs considerable formal research on portfolio performance for different categories of investment managers, including: newsletter writers, hedge fund managers, mutual fund managers and brokerage firm analysts. Guru Grades is just one corner of the site, examining one aspect of investing expertise.