Gurus
Can experts, whether self-proclaimed or endorsed by others (publications), provide reliable stock market timing guidance? Do some experts clearly show better intuition about overall market direction than others?
We have accumulated reviews of the public U.S. stock market forecasts of various investing/trading experts for more than two years. With nearly 5,000 measurements for about 60 gurus, including bulls and bears and technicians and fundamentalists, we have critical mass for: (1) assessing the forecasting acumen of the stock market gurus as a group; and, (2) ranking experts according to the accuracy of their past forecasts. This kind of forecasting ability is different from, but may be related to, stock picking expertise.
Note that the overall assessment of the stock market forecasting ability of experts in aggregate is far more reliable, based on sample size and duration, than the evaluations of individuals.
Note also that this study is not a test of whether the outputs of the experts are interesting, stimulating or useful in ways other than predicting the behavior of the overall U.S. stock market.
To interpret these grades correctly, please be sure to read our notes and the questions and answers section, both of which highlight our reasoning and address reader concerns.
We restrict reviews to publicly available material (freely available on the web), putting ourselves in the place of an individual investor trying to locate value in the marketplace, and mindful of concerns about copyright and trade secrets. Sometimes we find public records on the web sites of the experts themselves and sometimes on web sites of other parties (for example, the business media). This approach helps keep a level playing field for reviews, and it allows readers to check easily the context of forecasts and the reasonableness of our judgments.
The overall accuracy of the group based on both raw forecast count and on the average of forecaster accuracies (weighting each individual equally) is 47% (48%).
See the Notes section below for discussion of the grading process and for cautions on interpreting both the aggregate and the individual expert accuracy rates. See the Individual Gurus section below for links to the detailed evaluations for all these experts.
If you believe that past forecasting performance indicates future accuracy (skill, wisdom), you might want to keep tabs on the forecasts of the leaders. If you believe in reversals of fortune, you might follow those gurus with the lowest current accuracies. On the third hand, if you expect mean reversion in forecasting (pure luck), you might prefer the “Cone of Silence”. A fourth alternative is to apply Bayesian updating and weight the predictions of experts according to their evolving track records.
In summary, stock market experts as a group do not reliably outguess the market. Some experts, though, may be better than others.
For a simple analysis of the relationship between attention paid to individual gurus and their forecasting accuracy, see “Does Accurate Forecasting Get Attention?”. For another way to measure interest in individual gurus, see “The Most Intriguing Gurus?”.
For an unscientifically collected list of reasons why gurus go wrong, see “The Demon’s Defense”. For additional comic relief, see “Caught in the Too Fast Lane” and “Caught in Cash for an Entire Bull Market”.
Notes
Please note these points regarding the results in this table:
We selected experts to be graded based on extensive web searches for public archives offering at least marginal stock market forecast sample sizes. (Readers helped identify some commentators.) There may be data availability bias in the aggregation; some types of commentators may be more likely to offer frequent public commentary than others.
Many of the samples for individual experts are small, rendering confidence in the associated accuracies low. The aggregate sample, however, is large.
Assessing stock market forecasts requires judgment (ours) because they sometimes contain ambiguities, equivocations and/or conditional statements. We expect that our judgment errors tend to cancel each other, but we may have biases. Our detailed judgments are available for inspection via the links at the bottom of this page. Further, each individual review provides a link to source commentaries and articles, so that readers may decide whether the essential forecast language properly reflects commentary context.
When an expert commentary is too vague to assess or does not include a stock market forecast, we either do not include it or (if it is of background interest) we include but do not label it either “right” or “wrong.” For experts with small samples, we make a few exceptions for commentaries that include no market direction forecast but do offer some other significant and reasonably testable market-related prediction or recommendation.
When an expert issues weekly (monthly) commentaries, we tend to focus on the behavior of the market in the next week (month), unless the forecast specifies some other timeframe.
When judging whether a forecast is correct, we keep in mind empirical benchmarks based on weekly S&P 500 index data for January 1950-March 2008, as listed below. For example, if a guru says investors should be bullish over the next six months (26 weeks), and the market is up by only 2% over that interval, we would judge the call incorrect. Conversely, if a guru says investors should short the market over the next month, and the market is down only 1%, we would again judge the forecast incorrect (a losing position after trading and carrying costs). In summary, the grading process has normalizing or detrending effects such that the aggregate accuracy should probably be around 50%.
- About 50% of all one-week returns are greater than +0.1%.
- About 50% of all four-week returns are greater than +0.6%.
- About 50% of all 13-week returns are greater than +1.9%.
- About 50% of all 26-week returns are greater than +3.9%.
- About 50% of all 52-week returns are greater than +8.4%.
The tabular records of S&P 500 index performance for each individual guru are based on closing levels that commence as of the close on the forecast publication date. There is some looseness in this methodology because it does not take into account the publication timestamp of a forecast and the intraday (or opening) level of the index for that timestamp. The extreme case of looseness would be the different treatment of two forecasts timestamped just before and just after midnight (ET). In fact, many forecasters do not use timestamps. The grading of individual forecasts that focus on the short term takes into consideration this looseness in methodology
Assessments of different experts cover different timeframes according to the data available. This fact arguably weakens the case for ranking the experts. An expert who is stuck on bullish (bearish) would tend to outperform in a rising (declining) stock market. This effect should eventually cancel for the entire sample across all experts.
The private (for example, paid subscription) forecasts of experts may be timelier and more accurate than the forecasts recorded publicly. As noted above, we restrict reviews to publicly available material to: (1) maintain a level playing field for experts reviewed; and, (2) let readers check the reasonableness of our judgments.
With exceptions as noted, these assessments generally address forecasts of overall market direction, not the performance of stock picks. Most stocks, however, exhibit substantial co-movement with the overall market.
For further reading on the performance of experts in general, and what might distinguish a good market forecaster from a bad one, see the Investing Expertise category. Consider also also research on how individuals recollect and process advice as summarized in some of the items in the Individual Investing category.
Questions & Answers
A reader who is a financial advisor with a large investment services company sent a series of questions/comments regarding Guru Grades, as follows, with responses interleaved:
Question 1: “I’m guessing that you get paid by the gurus you hype, no?”
Response 1: No. Neither CXO Advisory Group LLC, nor any of its members personally, receive any payments or other compensation from the gurus reviewed on the site. The reviews are attempts to measure the ability of experts to forecast stock market behavior, not “hype.”
Question 2: “I’m flabbergasted by your top ‘gurus’, as I am familiar with them as well, and they are very poor performers. How can you tout a marketing company (Fisher) as a top money management company? Do you research these firms at all?????”
Response 2: The site does not “tout” any companies. It presents reviews of research, some models and some original analyses. The reviews and analyses presented are exactly as described on the site. It is up to readers to decide whether and how this information translates into action for them, or merely flabbergasts them.
Question 3: “Why do you compare what people say in articles/periodicals? It means nothing for professionals/investors! Their actual management/performance would certainly be more beneficial to ‘private investors and financial advisors’.”
Response 3: As stated above, the material cited addresses the following questions: “Can experts, whether self-proclaimed or endorsed by others (publications), provide reliable stock market timing guidance? Do some experts clearly show better intuition about overall market direction than others?” A reader suggested the concept of comparing multiple experts in this way some time ago. The discussion there further notes that: “This kind of forecasting ability is different from, but may be related to, stock picking expertise.” Results are not conclusive that any individual guru has special ability to forecast market behavior. The distribution of the accuracies of individual gurus about the mean is possibly normal. If the reviews of guru stock market forecasting records mean nothing to professionals/investors, then professionals/investors will pay no attention to them. Note that the Investing Expertise category catalogs considerable formal research on portfolio performance for different categories of investment managers, including: newsletter writers, hedge fund managers, mutual fund managers and brokerage firm analysts. Guru Grades is just one corner of the site, examining one aspect of investing expertise.
A reader suggested: “While your guru backtesting offers some fun in the way of balloon popping, it does not really offer much in terms of actual investment techniques. I would say the vast majority of your loyal audience is far more interested in [quantitative testing of trading rules] than in testing whether Guru X is or not another snake oil merchant. (Usually he/she is.) I have noticed that your tests of trading rules are widely cited in other places, but I have never seen anybody referencing your popping of another self-appointed guru. Reason? There is no surprise. We do know these guys offer zero. I humbly petition that you stop doing the Guru Grades.”
Response: There are two aspects to learning in all fields – what to do, and what not to do. In investing/trading, following the advice of gurus falls mostly into the second category. Many seasoned investors/traders understand that stock market gurus are, by and large, self-promoting and unable to provide accurate financial market forecasts and winning trades with significant reliability. However, other investors are in learning mode regarding guru expertise. Following the advice of experts is an investment technique. Hopefully, Guru Grades is not just fun but also helps these latter investors discover quickly how much attention (and money) they should devote to this technique. Reviews of log files indicate:
Blog entries on quantitative tests of trading rules and academic studies tend to draw a moderate number of readers immediately as posted and then withdraw to a quiet retirement, occasionally picking up a little traffic later via searches or links.
Blog entries on gurus tend to draw more readers initially and have more staying power, especially those covering well-known individuals such as Bob Brinker, Jim Cramer, Ken Fisher, Bill Fleckenstein and the Fast Money experts. Follow-up traffic again comes via searches and links from message boards. If you follow the “Site Hot Spots” feature in the right margin of site pages, you will see that several guru reviews (unlike any specific quantitative analysis or academic study) are consistently among the top 25 site pages in terms of unique visitors each month. This indicates a steady supply of readers seeking information on the credibility of guru pronouncements. The scope of the Guru Grades effort lends support to its conclusions. Nevertheless, the frequency of adding new gurus has dwindled as the number tracked has grown. It is a lot of work to compile a forecast/recommendation history.
Guru Grades is probably not of interest to all segments the CXOadvisory.com audience, but site traffic logs indicate that there is a fairly large segment with continuing interest in this section.
Two readers (one a tracked guru) have suggested that we drop David Nassar from the the list of gurus because: (1) his record is inactive, with no additional forecasts being added; and, (2) his record is relatively short (about 18 months).
Response: As stated, the objectives for Guru Grades are to answer the questions:
“Can experts, whether self-proclaimed or endorsed by others (publications), provide reliable stock market timing guidance?”
“Do some experts clearly show better intuition about overall market direction than others?”
Results indicate that, on average, experts do not provide accurate market forecasts. Dropping gurus from the study as they become inactive might introduce survivorship bias into these results. Several other gurus in the table are at least temporarily absent from the public forecasting forum. A bigger, longer study is probably necessary to determine whether level of accuracy relates to survival.
It is much more difficult to measure whether the dispersion of individual accuracies is random or derived from genuine ability. In either case, it appears that about two-thirds correct may be the best one should expect. David Nassar’s sample is relatively short in duration. However, his forecasting style appears to rely on short-term market trends (not broad bull or bear market views), such that there are a fair number of independent forecasts in his sample. In any case, individual reviews take sample size into consideration, and the discussion above includes several cautions regarding the scores of individuals, as follows:
“Note that the overall assessment of the stock market forecasting ability of experts in aggregate is far more reliable, based on sample size and duration, than the evaluations of individuals.”
“Many of the samples for individual experts are small, rendering confidence in the associated accuracies low. The aggregate sample, however, is large.”
“Assessments of different experts cover different timeframes according to the data available. This fact arguably weakens the case for ranking the experts. An expert who is stuck on bullish (bearish) would tend to outperform in a rising (declining) stock market. This effect should eventually cancel for the entire sample across all experts.”
A reader asked: “If you are not a subscriber to a service such as Prechters Elliott Wave then how can you judge them?”
Response: Guru Grades rates experts based on their public forecasts for the U.S. stock market, offered either systematically on their own web sites or as reported by commentary aggregators and financial media such as MarketWatch or Forbes. Use of public information allows transparency in the rating process by quoting forecasts and showing the grade for each forecast. Readers can adjust the subjective grades as they choose.
A subscriber to an investment advisory service can evaluate the accuracy of market forecasts provided by the service, but the terms and conditions of such services generally forbid redistribution of contents by subscribers. A third party therefore has great difficulty assessing the fairness/completeness of an evaluating subscriber’s grading method.
Should one have to pay for a forecasting service over many years to get a reasonably reliable reading of its accuracy? Free trials do not help much, because randomness dominates results for short measurement periods.
The primary objective of Guru Grades is to measure whether experts can accurately forecast the market. How much attention should investors/traders give to the pronouncements of gurus? The Guru Grades method suggests that experts on average offer little or no forecasting ability. A ranking of experts on forecasting accuracy is a less reliable output. While there is dispersion in the accuracy rates among individual experts, it is not obvious whether differences are due to skill (i.e., persistence over very long periods) or randomness. Sample durations, frequencies and time periods vary among rated experts. See, for example, “Converging Guru Accuracies”, which indicates that very long sample durations may be needed to detect true guru forecasting ability. The “Expert Political Judgment: How Good Is It? How Can We Know? (Chapter-by-Chapter Review)” summarizes an example of a very long study with some similarities to Guru Grades. Some of the lessons there likely apply to Guru Grades.
Note that there are alternative accuracy measurement systems for gurus, with priority on distinguishing among individual experts, such as (no affiliations):
TimerTrac, which states: “We Track Market Timing Professionals”. The gurus themselves must subscribe and submit calls to this service.
Hulbert Financial Digest/Hulbert Interactive, self-described as “a completely independent, impartial, and authoritative rating service that arms you with the facts about stock and mutual fund investment newsletter performance.” This service subscribes to investment newsletters and constructs portfolios that seek to mirror both timing and stock-picking advice. It does not evaluate other types of gurus.
A reader asked: “Can you adjust the guru performance track record considering detrending the market. I am thinking the S&P 500 Index would be a likely benchmark used for this analysis. Could your analysis include conclusions about both up market and down market performance in reference to the detrending adjustment? Also can the analysis ‘break out’ the gurus performance over the past 1 year, 3 years, 5 years and 10 years?”
Response: See the sixth item in the Guru Grades Notes above, which describes some basic detrending used as applicable in the Guru Grades grading process.
Although it would be very time-consuming because of the data structure, it would be possible to break down the Guru Grades forecasts into bull and bear markets. This segmentation would probably not change the aggregate accuracy rate of all gurus since they represent a mix of bulls and bears at any time. The aggregate inception-to-date aggregate accuracy rate has been roughly constant at a little below 50% for the life of the study. See the blog entries of “Guru Stock Market Forecasting Accuracy Over Time” and “Converging Guru Accuracies” for relevant analyses.
Breaking down the forecasts of individual gurus into bull and bear markets, in an effort to identify stuck clocks, is possible but would frequently result in subsamples much too small for credible inference. Just perusing the forecast records of individual gurus is a better way to identify stuck clocks.
Likewise, breaking down guru performance by calendar-based subperiods is possible but would result in many tiny subsamples. Also, many of the coverage periods for individual gurus are less than five years.
The introduction to Guru Grades provides the following caution: “Note that the overall assessment of the stock market forecasting ability of experts in aggregate is far more reliable, based on sample size and duration, than the evaluations of individuals.” Segmenting individual guru forecast records would exacerbate this reliability limitation.
In general, the Guru Grades study does not at this time lend itself to the detailed analyses suggested by your questions. The Guru Grades Notes address the study’s limitations.
A reader asked: “I was a little put off…to see someone that has proven to be borderline incompetent rated rather highly. The degree of accuracy of his incessant chirpy pronouncements is in the same vein as a stopped clock that’s right two times a day. Is there no way to measure the actual investment behavior of these ‘gurus’? What do they do with their own pronouncements? Ignore them or follow them?”
Response: Without knowing which “chirpy” guru you mean, there is no way to comment on your “borderline incompetent” assessment.
Sometimes it is possible to measure actual (or hypothetical) investment performance, such as when a guru runs a publicly traded fund, clearly specifies an investing/trading methodology or publishes a meaningfully long archive of past recommended trades. See, for example:
- Cramer Offers You His Protection?
- Robert Drach, Trading with 95% Confidence?
- Lenny Dykstra at Bat
- Stock Picking Performance of Fast Money Experts
- Forbes Evaluates Ken Fisher’s Stock Picking
- The Timing Value of John Hussman’s Market Climate Assessments
- Testing Navellier’s Stock Picking and Market Timing Based on Fund Performance
- Steve Sarnoff’s Advice at the “Options Hotline”
- Analysis of James Stewart’s “Common Sense” Stock Market Timing Strategy
- “Strategy Lab” Performance
- Steve Todd’s Intermediate-Term Market Calls, A Forward Test
- Safe with Martin Weiss?
There are other web sites/services (no affiliations) that attempt to simulate or track guru investment performance, such as Hulbert Financial Digest (paid service covering newsletter writers only), GuruFocus and TimerTrac (voluntary only). However, many gurus are “black box” commentators with no substantial evidence of performance (subscriber standards for evidence of actual performance seem low).
There is also a stream of formal research that examines the investing performance of experts, usually in aggregate (browse the Investing Expertise category). Some items in the Individual Investing category are also relevant.
It is hard to tell whether gurus ignore or follow their own pronouncements. Some gurus may be earnest in their advice and may attempt to follow it themselves. Others may be hard-boiled cynics who know how to get people to pay them for advice of unknown value. See “What About Dan Murphy?” and “Mark Skousen’s Claims So “Wild” They Might Be True?” for cautionary (and amusing) examples.
A reader asked: “For Guru Grades, why don’t you create net asset values of investment advice history (as you do for “The Decision Moose Asset Allocation Framework”)? In addition to being relevant (we do not care about the hit ratio but about the bottom line!), such modeling would also show payoff profiles and give the true credit to the bears who lose a little very often but win big in a few occurrences (like buying puts).”
Response: The methodology of Guru Grades does not fit your suggestion for many of the gurus. Specifically, irregular (e.g., media-covered) sampling of the “black box” forecasting outputs does not enable plausible modeling of a guru’s portfolio performance.
Guru Grades addresses the question of whether gurus can reliably forecast U.S. stock market behavior; it does not measure whether they can outperform the stock market based on portfolio returns (many other studies and sources measure the latter). In fact, the gurus covered generally offer no convincing record of their investing performance. The connection between forecasting ability and investing performance is imprecise, and many investors/traders may not care about the study. See the prior question and response for additional discussion regarding when return modeling is possible and when it is not.
The results of Guru Grades should arguably make consumers of expert advice doubtful about guru claims of materially exploitable foresight. Should consumers of financial markets expertise accept obliquely implied value, vague defenses and strong disclaimers, which apparently comprise the resolution of “The Demon’s Dilemma” for many gurus?
Would not any gurus who could present convincing records of reliable market outperformance, whether built on many good small bets or just a few good big bets, enthusiastically make their complete real investing records public? If so, what should consumers of expertise conclude about the absence or vagueness of public investing records for many self-proclaimed or media-promoted gurus?
If there are quantitative public investing records for any “bears who lose a little very often but win big in a few occurrences,” and thereby reliably beat the market over long periods, these records would merit review.
A reader commented and asked: “Where do you get your ‘publicly available’ information on which to grade ‘guru’ Jason Kelly? I have been a subscriber of his since at least six months before the meltdown, and he has made all the wrong choices. [Specific examples withheld since they are likely proprietary.] His results are atrocious. So how do you get that he is ‘essentially right’ to the degree that you report in Guru Grades? Following his investment advice has been probably the worst disaster of my life, financially!”
Response: As described in “Jason Kelly: The Neatest Little Market Advice?”, the public forecasts of Jason Kelly come from commentary at his web site. His overall accuracy rate is currently 60% correct (40% incorrect). The specific forecasts graded and forecast-by-forecast grades are available for review in the table at the bottom of the evaluation.
Reasons why your experience differs from the bottom-line result at Guru Grades could include:
- Jason Kelly’s private forecasts, or their implementations, are less accurate than his public ones.
- The judgment applied in grading the public forecasts is defective.
- Your experience with Jason Kelly’s advice (2008-2009) is short compared to the graded forecast sample (late 2001-2009). Since the beginning of 2008, his forecast accuracy rate is only 31% correct (subsample of just 14 graded forecasts), much lower than his overall average.
Reason 3 stands out as a likely explanation. Jason Kelly’s public forecasting record since the beginning of 2008 does not seem to be in conflict with your experience.
A reader commented: “I was looking over your guru grade for Robert Prechter. While I haven’t been following this guru’s advice over the extended period of time that your chart represents, I have followed his newsletters over the last two years. The advice that he gave over the last two years both in his newsletters and on Bloomberg is not accurately represented by your table. There are several missing calls, and the ones that are there are not correct. Specifically:”
- “What I immediately see is that MarketWatch has entries from every month or two from 2002-2005, but nothing from 2005-2008. Then the entries pick up again in 3/09. This is extremely inconsistent, and this lapse in time misses what amounts to a hugely accurate time period for Prechter.
- “Maybe MarketWatch just gave up listening to Prechter for a few years because he was bearish for the whole rally from 2003 on up. I would understand that, but still, this is a study, and it needs to have consistent entries to be of any value.
- “Additionally, the comments Marketwatch chose to put into the data table look like they are both poor in choice and out of context.
- “The last issue I have is that in MarketWatch’s additional notes, which are above the table, there is an article from 3/4/2009 by Peter Brimelow, in which he recognizes that Robert Prechter recommended a fully leveraged short position on 7/17/2007, entering what obviously became a grand-slam trade. Prechter then recommended closing the position in February 2009, having been on the right side of the market from 13,000 to 6500 in the DJIA. Where is this in the table?
- “If they had updated the table more consistently, Marketwatch would have a few more negative numbers on the S&P returns. It’s so incomplete that the largest drop in the S&P recorded during this period is -7.1% ! How skewed is that? At one point, the market was down more than 50% while Prechter was short.
- “The story and data table presented from Marketwatch’s comments could not be more misleading during this time period.
- “Your table…needs updating to be accurate.”
Response: You are probably right that the media tends to stop quoting experts after they have been incorrect over long periods. Conversely, the media probably tends to start quoting experts after they have made some dramatically correct call. It is not obvious that such uneven attention biases long-term measurement of stock market outlooks reported by the media. One might conclude that Robert Prechter, because of a persistent bearishness, tends to be dramatically correct during bear markets and dramatically incorrect during bull markets. Since bear markets have generally been much shorter than bull markets, a regularly sequenced sample of his forecasts would probably find him mostly incorrect.
Excerpting of the comments for the table within “Robert Prechter: 100-Year Bear?” is at the discretion of CXOadvisory.com, not MarketWatch.com, so the blame for any poor choices or lack of context belongs here. However, readers can readily locate original articles and judge for themselves (by searching “Prechter” at MarketWatch.com). The excerpts in the table are those referencing Robert Prechter’s personal beliefs (not those expressed only by his associates or by newsletters written by his associates).
The augmenting quotes above the table offer another perspective on the usefulness of Robert Prechter’s methods based on information made publicly available from the Hulbert Financial Digest, which maintains model portfolios based on the investing advice of a wide range of newsletters. Again, excerpting for review purposes is at the discretion of CXOadvisory.com. Readers can readily link to source articles.
The Bloomberg.com archive of Robert Prechter’s comments is fairly short and highly repetitive (generating multiple citations for each comment).
Note that “Robert Prechter: 100-Year Bear?” states:
“…because of his very long forecasting horizon, the sample is much too small for reliable inference.”
“Because the sample of reported forecasts for Mr. Prechter is so small, we are not including him in the Guru Grades snapshot.”
For other cautions about the Guru Grades methodology, see Guru Grades “NOTES”.
Media citations are often not the best way to measure expert forecasting accuracy. Some readers may find such surveys of no value. Other readers may take them as an indication of how much they should pay attention to reports of expert opinions on the future behavior of asset markets. In this case, much uncertainty and debate could be eliminated if Robert Prechter would make publicly available his past newsletter commentaries or (especially) his actual long-term investing performance.
A reader comments and asks: “Guru Grades has 4586 “measurable forecasts” as of 1/1/2010 and only 2199 are graded “essentially right.” The Excel formula <=BINOMDIST(2199, 4586, 50%, FALSE)> indicates that there is only a 0.29% chance that the gurus could have forecasted worse, given a 50-50 coin-flip ratio of forecasts being reviewed as right or wrong. There are three possible explanations of this strikingly low probability:
- The gurus on your list are extraordinarily poor forecasters.
- You are biased and tend to punish the forecasters.
- The forecasters may be poor at calling the direction, but their ability to forecast magnitude (i.e., how large a market swing will be) may be better, and this ability has not been reviewed, thus indicating that your guru list is of little interest for real-life investors whose returns are determined by hit ratio times magnitude of market swing.
If reality is a mix of these three points, it could be that the guru stats are of no statistical significance. Have you considered your guru stats in light of these points?”
Response: The interpretation of the statistical output you calculate is that the probability of finding a 48% accuracy rate in such a large sample of independent binary measurements, if the actual accuracy rate is 50% (no forecasting ability at all), is very small. Note that guru forecasts are not generally independent, in that gurus tend to develop forecasts based on information from intervals much longer than the average interval between their forecasts (they use the same information more than once), likely impounding serial correlation into forecasts. Because of messiness such as this likely serial correlation, effective sample size is likely much smaller than the simple number of graded forecasts.
With respect to identification of potential biases, see Guru Grades “NOTES” (especially the first, third and sixth items) and several of the above questions and responses. See also “THE DEMON’S DEFENSE”.
Here are two additional considerations not explicitly addressed in the above citations that may systematically affect guru forecasting accuracy:
- The period covered by the forecast sample in aggregate is an historically bad decade for the U.S. stock market. This anomalous performance might have surprised experts on average (whose beliefs are at least partly anchored in prior decades).
- As implied by the first item in Guru Grades “NOTES”, gurus who make frequent public forecasts may be those most prone to issuing extreme forecasts to attract attention by stimulating greed and fear. See “Why Gurus Go to Extremes” and “A Sign of All Times…”.
You may also be interested in the broad collection of items in the Investing Expertise category.
