Guru Grades

"Strategy Lab" Performance (Last Updated 7/28/08)

We review here the performances of 25 professional and five amateur investors/traders who have participated in 11 rounds of MSN Money's "Strategy Lab" since late 2001. "Strategy Lab" involves a series of multi-month tests of the investment strategies of six participants at a time. The rules "give each trader a hypothetical $100,000 portfolio to manage. The traders are free to deploy the cash using their unique strategies as they see fit. Some might choose to invest all of their money on the first day. Some might never be fully invested... Before the strategists can make a move in their portfolios, they have to tell you about it in their journal entries." Here's how they have done:

Results comprise a total sample of 66 investing/trading records for:

• Round 7 (12/3/01-10/11/02)
• Round 8 (1/11/03-8/15/03)
• Round 9 (8/26/03-3/11/04)
• Round 10 (3/18/04-10/14/04)
• Round 11 (10/22/04-2/25/05)
• Round 12 (4/28/05-10/21/05)
• Round 13 (10/31/05-6/1/06)
• Round 14 (6/16/06-12/21/06)
• Round 15 (1/8/07-6/28/07)
• Round 16 (7/16/07-1/18/08)
• Round 17 (1/28/08-7/25/08)

The following chart depicts the average performance of participants by round, along with the concurrent performance of the value-weighted S&P 500 index for comparison. Other indexes may be more appropriate benchmarks for some participants, depending on investing style. Participants in aggregate have beaten the S&P 500 index five rounds out of 11.

The next chart depicts the ranges of participant returns by round, with returns for the S&P 500 index indicated for reference. The average range of participant returns across all rounds is about 35%. This large variability demonstrates that different stock selection rationales produce very different results.

Other points of interest are:

• Participants individually outperform the S&P 500 index 40 times (61%) and underperform 26 times (39%). The seven amateur instances are three outperforms and four underperforms. Again, different benchmarks may be better suited for some participants.
• Across all rounds, participants average a 3.5% gain per round, compared to 0.9% for the S&P 500 index. However, a sample of 11 is very small, and one new observation could substantially change results. For example, if we start with Round 8 rather than Round 7, the average participant return per round is 3.7% and the average S&P 500 index return per round is 3.0%.
• Across all 11 rounds, the correlation between the average participant return in a round and the return on the S&P 500 index for that round is 0.61. The R-squared statistic is 0.37, indicating that overall market returns determine 37% of participant returns. Excluding Round 7 boosts the R-squared statistic to 0.57 (with the large change indicative of a very small sample).
• Some participants have complained about the suboptimizing effect of the Strategy Lab announcement rule that prevents them from trading quickly.

In summary, the portfolio returns of "Strategy Lab" participants since late 2001 vary widely, but aggregate results offer some support for the belief that experts on average consistently outperform the broad stock market.

See Guru Grades for a snapshot of the accuracy of various experts in predicting the direction of the U.S. stock market, including links to evaluations of their individual commentaries. See also Blog Synthesis: The Wisdom of Analysts, Experts and Gurus for more general research on the performance of expert investors.

To discuss this review, go to the Guru Grades Forum.