How well does the Morningstar fund rating system (one star to five stars) work? In their November 2016 paper entitled “The Morningstar Rating for Funds: Analyzing the Performance of the Star Rating Globally”, suggested for review by a subscriber, Jeffrey Ptak, Lee Davidson, Christopher Douglas and Alex Zhao analyze the global performance of star ratings in terms of ability to predict fund performance. They use two test methodologies:
- Monthly two-stage regressions that test the ability of fund star ratings to add value to a linear factor model for each asset class at a one-month horizon. The first stage estimates fund dependencies (betas) on commonly used predictive factors over the past 36 months. The second stage measures the ability of fund star ratings to add predictive power to those betas in the following month. For stock funds, they consider fee, equity market, size, value and momentum factors. For bond funds, they consider fee, credit and term factors. For stock-bond funds, they consider all these factors. For alternative asset class funds, they consider fee and equity market factors.
- An event study that tracks performances of equally weighted portfolios of funds formed by prior-month star rating over the next 1, 3, 6, 12, 36 and 60 months.
Using fund categories, monthly fund star ratings and returns, and asset class factor returns during January 2003 through December 2015, they find that:
- From regression tests (see the table below):
- For stock funds, star ratings exhibit some but not conclusive incremental predictive power. The monthly premium of 5-star funds over 1-star funds is 0.09% (1.03% annualized).
- For bond funds, star ratings exhibit significant incremental forecasting ability. The monthly premium of 5-star funds over 1-star funds is 0.09% (1.09% annualized).
- For stock-bond funds, star ratings exhibit significant incremental forecasting ability. The monthly premium of 5-star funds over 1-star funds is 0.15% (1.75% annualized).
- For alternatives funds, star ratings do not exhibit reliable incremental forecasting ability. The monthly premium of 5-star funds over 1-star funds is 0.04% (0.48% annualized).
- Regarding the event study (see the chart below):
- For stock funds at a three-year horizon, 5-star funds on average outperform 1-star funds by an annualized 0.64%.
- For bond funds at a three-year horizon, 5-star funds on average outperform 1-star funds by an annualized 0.35%.
- For stock-bond funds at a three-year horizon, 5-star funds on average outperform 1-star funds by an annualized 0.59%.
- For alternative funds at a three-year horizon, 5-star funds on average outperform 1-star funds by an annualized 0.17%.
- Persistence of the performance gap between high-rated and low-rated funds likely derives from the strong inverse correlation between star ratings and fees.
The following table, extracted from the paper, summarizes annualized differences in premiums (percentages) between 3-star funds and each of 1-star, 2-star, 4-star and 5-star funds by category as generated by the regression methodology. For equity, bond (fixed income) and stock-bond (balanced) funds, 1-star and 2-star funds consistently underperform 3-star funds, and 4-star and 5-star funds consistently outperform 3-star funds, and results are systematic across star ratings. For alternative funds, however, results are not systematic and 1-star and 2-star funds outperform 3-star funds (3-star funds are the worst performers).
The following chart, taken from the paper, summarizes event study performance by star rating across all fund categories. Higher-star funds systematically and consistently outperform lower-star funds, but the performance differences are small.
In summary, evidence indicates that the Morningstar star rating system has some ability to predict risk-adjusted returns in the short term and does more good than harm to investors over the long term.
Cautions regarding findings include:
- Per the December 2013 “Morningstar Methodology Paper”, the last “significant” changes to the star rating methodology occurred in 2006, more than three years after the beginning of the sample period. That paper further states: “In addition to the [significant] changes, Morningstar has occasionally added new categories to the category structure in each market and those changes can also impact the fund ratings.” One of the authors of the paper reports that post-2003 methodology changes are not material to the study.
- As noted in the paper, the regression methodology is short-term (one month ahead) and may not reflect long-term investor experience.
- As noted in the paper, the event study does not control for fund risks.
- Reliably exploiting findings requires (impractically) holding a portfolio of many mutual funds with periodic reformation.
- Applying the test methodologies to multiple fund subsamples (categories) introduces snooping bias, such that the strongest findings overstate expectations.
Frequently Asked Questions
What is the Morningstar star rating system for mutual funds?
The Morningstar star rating system grades funds from one star to five stars, with higher ratings intended to signal better past risk-adjusted performance relative to peers in the same category. The 2016 study of the system examined its global performance as a predictor of future fund returns across stock, bond, stock-bond and alternative funds.
Do Morningstar 5-star funds actually outperform 1-star funds?
Morningstar 5-star funds outperformed 1-star funds in the January 2003 to December 2015 tests, though the differences were small. The annualized premium was 1.03% for stock funds (not conclusive), 1.09% for bond funds and 1.75% for stock-bond funds, while alternative funds showed no reliable predictive ability (0.48%). Overall, the star ratings showed some ability to predict short-term risk-adjusted returns.
How do researchers test whether star ratings predict fund performance?
Researchers tested Morningstar star ratings with two methods over January 2003 to December 2015. The first was a monthly two-stage regression that estimates fund betas on factors such as fees, equity market, size, value, momentum, credit and term over 36 months, then tests whether star ratings add predictive power the following month. The second was an event study tracking equally weighted portfolios of funds grouped by prior-month star rating over 1, 3, 6, 12, 36 and 60 months.
Why do low-fee funds tend to get higher Morningstar ratings, and does that explain the performance gap?
Star ratings and fees are strongly inversely correlated, and the study suggests this likely explains the persistence of the performance gap between high-rated and low-rated funds. For stock, bond and stock-bond funds, 4- and 5-star funds consistently outperformed 3-star funds and 1- and 2-star funds underperformed them. For alternative funds the pattern was not systematic, with 1- and 2-star funds outperforming 3-star funds.
What are the limitations of using Morningstar star ratings to pick funds?
The Morningstar star ratings study carries several caveats: the regression tests look only one month ahead, the event study does not control for fund risks, and Morningstar's methodology last changed significantly in 2006, within the sample period. Testing multiple fund categories introduces snooping bias, so the strongest findings likely overstate expectations. Exploiting the results would also require impractically holding many funds with periodic reformation, and the performance differences were small.