As the famous quote by Prime Minister Benjamin Disraeli states: “There are three kinds of lies – lies, damned lies, and statistics”. In this installment of Financial TIPs we will focus on the third type of “lie”, statistics. One trick of the trade we must watch for when accepting or rejecting a performance analysis is survivorship bias; using statistics to mislead.
What Is Survivorship Bias?
Only the strong survive. This is a familiar adage because it’s often true – especially in our financial markets. That’s why it is important to remember the expression whenever we want to accurately assess a sample of past returns. Examples of a “sample” might be the returns from all actively managed U.S. stock funds during the past decade, or the returns from all global bond funds from 2000–2014.
Survivorship bias occurs when an analysis omits returns from in-sample funds that were closed, merged into other funds, or otherwise died along the way. Do not get caught in the cognitive illusion that makes an appearance whenever failures disappear or become hidden from view as described in this Being Human article titled The Pitfall of Studying (only) Success.
How Often Do Funds Go Under?
Some new funds are truly innovative, do well by their investors, and become familiar names. Less-sturdy ones may instead focus on trying to seize and profit from popular trends. For these, the expression “cannon fodder” comes to mind. They may (or may not) soar briefly, only to fizzle fast when popular appeal shifts.
In the competitive capital markets in which we operate, fund managers launch new products and discontinue existing ones all the time. Individual funds probably disappear far more frequently than you might think.
- A recent S&P Dow Jones Indices analysis found that, for the five-year period ending in December 2015, “nearly 23% of domestic equity funds, 22% of global/international equity funds, and 17% of fixed income funds have been merged or liquidated.”
- As might be expected, the longer the timeframe, the higher the death rate. A January 2013 Vanguard analysis of survivorship bias looked at a 15-year, 1997–2011 sample of funds identified by Morningstar. The analysis found that 46 percent “were either liquidated or merged, in some cases more than once.”
- A May 2015 Pensions & Investments (P&I) article reported that Exchange-Traded Products (including ETFs) weren’t immune from the phenomenon either, having just reached the milestone of 500 products closed. According to the “ETF Deathwatch” cited source, this represented a mortality rate of just under 23 percent.
- A November 2015 article by financial columnist Scott Burns found similar survival rates for the 15-year period ending in 2014. “At the beginning of the period, there were 2,711 funds,” he reported. “At the end of the period, there were 1,139. Only 42 percent of the starting funds had survived.”
Why Does Survivorship Bias Matter?
Why should you care about the returns of funds that no longer exist?
In Dimensional’s analysis of US Mutual Fund Landscape investors may be surprised by how many mutual funds both equity and fixed income become obsolete over time. Funds tend to disappear quietly, and underperforming funds—especially those that do not survive and are no longer available for investment—receive little attention.
The funds that disappear from view are usually the ones that have underperformed their peers. The aforementioned Vanguard analysis found that, whether a fund was liquidated or merged out of existence, underperformance was the common denominator prior to closure.
If these disregarded data points were athletes on a professional sports team, they’d be the ones bringing down their team’s averages. When assessing a team’s overall performance, it’s important to consider both the wins and the losses, right? Same thing with fund performance.
Instead, an analysis marred by survivorship bias is highly likely to report overly optimistic outcomes for the group being considered. While a degree of optimism can be admirable in many walks of life, basing your investment decisions on artificially inflated numbers is more likely to set you up for future disappointment than to position you for realistic, long-term success.
Moreover, survivorship bias is only one of a number of faults that can weaken seemingly solid reports. One way in which we strive to add value to investors’ investment experience is to help them separate robust data analysis from misleading data trickery. We hope you’ll be in touch if we can assist you with your own strategies and selections in a market that is too often rigged against the individual investor.