We review here the performances of 27 professional and six amateur investors/traders who have participated in 12 rounds of MSN Money’s “Strategy Lab” since late 2001. “Strategy Lab” involves a series of multi-month tests of the investment strategies of six participants at a time. The rules “give each trader a hypothetical $100,000 portfolio to manage. The traders are free to deploy the cash using their unique strategies as they see fit. Some might choose to invest all of their money on the first day. Some might never be fully invested… Before the strategists can make a move in their portfolios, they have to tell you about it in their journal entries.” Here’s how they have done:
Results comprise a total sample of 66 investing/trading records for:
- Round 7 (12/3/01-10/11/02)
- Round 8 (1/11/03-8/15/03)
- Round 9 (8/26/03-3/11/04)
- Round 10 (3/18/04-10/14/04)
- Round 11 (10/22/04-2/25/05)
- Round 12 (4/28/05-10/21/05)
- Round 13 (10/31/05-6/1/06)
- Round 14 (6/16/06-12/21/06)
- Round 15 (1/8/07-6/28/07)
- Round 16 (7/16/07-1/18/08)
- Round 17 (1/28/08-7/25/08)
- Round 18 (8/1/08-2/13/09)
The following chart depicts the average performance of participants by round, along with the concurrent performance of the value-weighted S&P 500 index for comparison. Other indexes may be more appropriate benchmarks for some participants, depending on investing style. Participants in aggregate have beaten the S&P 500 index six rounds out of 12.
The next chart depicts the ranges of participant returns by round, with returns for the S&P 500 index indicated for reference. The average range of participant returns across all rounds is about 37%. This large variability demonstrates that different stock selection rationales produce very different results.
Other points of interest are:
- Participants individually outperform the S&P 500 index 45 times (63%) and underperform 27 times (38%). The seven amateur instances are five outperforms and four underperforms. Again, different benchmarks may be better suited for some participants. Relative performance is a combination of market timing and stock picking. In general, when participants do not fully deploy all cash, they tend to underperform (outperform) a rising (falling) market.
- Across all rounds, participants average a 2.0% return per round, compared to -2.0% for the S&P 500 index. However, a sample of 12 is very small, and one new observation could substantially change results.
- Across all 12 rounds, the correlation between the average participant return in a round and the return on the S&P 500 index for that round is 0.75. The R-squared statistic is 0.56, indicating that overall market returns determine 56% of participant returns. Excluding Round 7 boosts the R-squared statistic to 0.66 (with the large change indicative of a very small sample).
- Some participants have complained about the suboptimizing effect of the Strategy Lab announcement rule that prevents them from trading quickly.
In summary, the portfolio returns of “Strategy Lab” participants since late 2001 vary widely, but aggregate results offer some support for the belief that experts on average can consistently outperform the broad stock market.
As of 2/13/09, MSN Money states that “‘Strategy Lab’ is going on hiatus,” with no new rounds scheduled.
See Guru Grades for a snapshot of the accuracy of various experts in predicting the direction of the U.S. stock market, including links to evaluations of their individual commentaries. See also Blog Synthesis: The Wisdom of Analysts, Experts and Gurus for more general research on the performance of expert investors.
Frequently Asked Questions
What is MSN Money's Strategy Lab?
Strategy Lab was an MSN Money series of multi-month tests in which six investors or traders at a time each managed a hypothetical $100,000 portfolio using their own strategies. Participants had to disclose their moves in journal entries before making them. As of 2/13/09, the series was on hiatus with no new rounds scheduled.
Do professional investors in Strategy Lab beat the S&P 500?
Strategy Lab participants on average outperformed the S&P 500 index, returning 2.0% per round versus -2.0% for the index across 12 rounds from 2001 to 2009. Individually, participants beat the index 45 times (63%) and underperformed 27 times (38%), though in aggregate they won only six of 12 rounds. The article says results offer some support for experts consistently beating the market, but the sample of 12 rounds is very small.
How was the performance of Strategy Lab investors measured against the market?
Strategy Lab performance was measured by comparing each participant's return on a hypothetical $100,000 portfolio over a multi-month round with the concurrent return of the value-weighted S&P 500 index. The sample covers 27 professional and six amateur investors across 12 rounds, totaling 66 investing/trading records. Results were examined as round averages, ranges of participant returns, and the correlation between average participant return and the S&P 500 return (0.75, R-squared 0.56).
Do stock market returns explain most of an active investor's performance, and how does holding cash affect results?
Overall market returns explain about 56% of Strategy Lab participant returns, based on an R-squared of 0.56 and a 0.75 correlation with the S&P 500 index. Excluding Round 7 raises R-squared to 0.66. Relative performance combines market timing and stock picking, and participants who do not fully deploy cash tend to underperform a rising market and outperform a falling one.
What are the limitations of using Strategy Lab results to judge expert investors?
Strategy Lab results rest on a very small sample of 12 rounds, so one new observation could substantially change the conclusions; excluding Round 7 alone moved R-squared from 0.56 to 0.66. The S&P 500 may not be the right benchmark for every participant's investing style, and participants vary widely (average range of returns across rounds is about 37%). Some participants also complained that the rule requiring them to announce moves before trading prevents quick trading and suboptimizes results.