Thanks. W/re to the critique that a 'gap' is a red herring because markets are zero sum (to paraphrase), yes if you were to run this study across all markets as a whole then it should approximate zero. We are focused on the US mutual fund market, a subset of the global market.
W/re to the critique that our estimates of the gap could misrepresent the dollar-weighted return of individual investors - yeah that's true and that's why we have never positioned the study in those terms to my knowledge. For instance, we do not state 'The average investor earned x% while the average fund earned x+1.50%'. We also do not refer to it as a 'behavior gap' as the author has. We are attempting to estimate the return of the average dollar invested in US funds and ETFs. In doing so, we pool the beginning assets, monthly net flows, and ending assets of *all* funds and ETFs that existed at the start of our 10-year study period.
Notwithstanding that, the author seems to imply an investor who temporarily left the hypothetical index fund earned zero return in the interim as far as our study's methodology is concerned. That isn't accurate. We would capture the outflow from that index fund and then if that investor put the proceeds in a different fund that's part of our study universe (which the author's example doesn't imagine, as it's a one-fund universe in that example) we'd capture the inflow to that other fund and then when that investor redeemed from that other fund to return to the index fund we'd capture the outflow from that other fund and the inflow to the index fund, respectively.
But let's suppose our study was that single hypo index fund the author presents and we are trying to estimate the return of the average dollar in that hypo fund. In year one, the two investors put down $20k in aggregate and finished the year with $21k, which they pulled out entirely. In year two, since there was no money in the fund, its return is irrelevant. In year three, the two investors put in $23,100 (which assumes the first investor sat in cash, earning nothing, and the other investor, glass half full, earned 20% in wherever he put the year 1 proceeds, though it's not clear from the example if we're talking about the same monies vs., glass half empty, him plunking down more cash) and earned 5% on that to finish with $24,255. The average dollar in that pool of capital--the index fund--earned a 5% IRR. That's what we're attempting to estimate.
The author might object to us saying that the average dollar in his hypo universe--the one fund--earned 5% despite the fact that the second investor earned substantially more. But that second investor either exited our universe or put additional monies in. We don't know. (In the real world, ie, our study where it's not a single fund rather thousands, it's likely we capture that investor's contribution to the other fund where she made hay and then redeemed to put the enlarged sum to work in the index fund in question.)
1. You say, “…yes if you were to run this study across all markets as a whole then it should approximate zero. We are focused on the US mutual fund market, a subset of the global market.”
ME response: Then are you saying that if you focus on the US mutual fund market, you get a positive gap (i.e., US mutual fund investors underperform their investments), while if you focus on the whole world, you have to get approximately zero? That would imply that when US mutual fund traders compete with other traders, they suffer by comparison – presumably by timing the market badly. Do you believe that, and if this is really evidence for that (which I don’t believe), is there any other corroborating evidence? Why would US mutual fund traders specifically be worsted by those they trade with?
2. You say, We “do not refer to it as a 'behavior gap' as the author has.”
ME response: But surely you are aware that that is exactly how virtually everybody else in the investment field is interpreting your measure. And furthermore, Morningstar’s pronouncements on its web sites very clearly imply that this is what the measure shows. Take this web page for example: https://global.morningstar.com/en-gb/funds/buy-and-hold-or-time-the-market-mind-the-gap. Shouldn’t you enlighten them that they should not interpret it that way? And if you do, shouldn’t it be presented as a correction to your previous pronouncements? And if they should not interpret it that way, what interpretation should they put on it, if any?
3. You say, “the author seems to imply an investor who temporarily left the hypothetical index fund earned zero return in the interim as far as our study's methodology is concerned.”
ME response: No, I do not imply that. I merely assumed that investor A in my example did earn a zero return in the interim while investor B did not. I did not assume that all investors earned a zero return in the interim.
4. You say, “We would capture the outflow from that index fund and then if that investor put the proceeds in a different fund that's part of our study universe … we'd capture the inflow to that other fund and then when that investor redeemed from that other fund to return to the index fund we'd capture the outflow from that other fund and the inflow to the index fund, respectively.”
ME response: This is news to me. To the best of my knowledge your description of the “mind-the-gap” measure does not portray it this way. And anyway, how could you do that unless you knew exactly where that particular investor was placing her funds at all times? And that would only be for one investor. Of course there are individual investors who happen to time the inflows and outflows to their mutual funds wrong, but then there must be others on the other sides of their trades who do it right. But you can’t track the investments of a whole category of investors the way you just described in such a way as to claim (as I’m afraid you do) that a whole group of them exhibits a “behavior gap.”
5. You have two paragraphs beginning with, “But let's suppose our study was that single hypo index fund the author presents…etc.”
ME response: I’m sorry but I don’t fully understand those paragraphs. Perhaps you can clarify further.
Thanks. Some thoughts which I've numbered to correspond to your responses above:
1. As you know, we lack the data to estimate the return of the average dollar in realms that are less transparent/measurable than the U.S. fund market. It is axiomatic that if markets are zero sum, if we are observing a shortfall to total return in one area (US funds), there would have to be an offsetting surplus elsewhere (ex US funds). As to why that would be so, information asymmetry; scale advantages; transaction frictions; etc. I do not have evidence for this, though I would imagine there have been studies of this in the academic literature. Does that call the possibility of a gap's existence into question? I suppose you could argue that (which you have) but I'm guessing that if we found that, say, the time-weighted pre-fee return of a particular group of funds diverged from that of a representative benchmark, we would not question its validity because we can't empirically find the offset elsewhere?
2. You're correct that in the past we referred to it as a 'behavior gap' and I think others who have done research in this area have used similar terms. However, we have made a concerted effort in recent years to avoid that term because it evokes the notion of people acting rashly, etc. when in reality dollar-weighted outcomes can vary owing to context and other circumstances. A good example is smaller gaps among allocation funds, that likely owing to such funds being heavily used in the 'gilded cage' context of a retirement plan that encourages long-term saving and discourages transacting mainly by automating routine tasks like rebalancing, etc. Conversely, narrower vehicles like sector funds, which are less commonly found in that context, have tended to bedevil investors.
To make this more explicit, page 3 of the study states: "To be sure, inopportunely timed purchases and sales—buying high or selling low on impulse, for instance—can chip away at investor returns. But even laudable practices like investing a portion of every paycheck or regularly rebalancing can open a gap between investor results and reported total returns. Given that nuance, it's not advisable to view this study's findings as a parable of "dumb money" or evidence of individual investors' fallibility."
3. We'll agree to disagree on this point I guess. I think that your example--which I realize is presented for simplicity's sake and that's fine--is misleading in suggesting that the study is so truncated as to omit large swaths of activity. The study encompasses more than 25,000 funds representing the lion's share of the industry's assets. As I mentioned, we pool beginning assets, monthly flows, and ending assets, thereby attempting to capture movements between funds that are a part of our study universe and also including funds that have been merged or liquidated away.
4. We lay out our methodology in the report (it's in the appendix). It has been clearly disclosed for some time now and I'm not familiar with the description you're attributing to us ("To the best of my knowledge your description of the “mind-the-gap” measure does not portray it this way."). We are not attempting to estimate the dollar-weighted return of particular investors. Rather, we are trying to estimate the return of the average dollar in our pool. To avoid creation bias, we limit the study to funds that existed at the start of our study period, including funds that subsequently died.
5. I was imagining your hypo example was the 'US fund industry' and then applying the methodology that we use in conducting our study, whereby we pool the beginning assets, periodic flows, and ending assets so as to derive the IRR. As mentioned, I think it's an oversimplistic construct given that it imagines us being blind to a year's activity (if it's a one-fund universe and both A and B do not capture the return of that one fund in year 2 then it effectively means they went elsewhere and thus the study is blind to 1/3 of the starting investors' activity). But mainly I wanted to explain how the study would approach it methodologically.
1. Yes, if the time-weighted pre-fee return of a particular group of funds diverged negatively from that of a representative benchmark (i.e. the total market) then you could say that those funds underperformed the market, but not that the investors in those funds underperformed their investments – only that they chose the wrong ones. And you could conclude that all the other funds therefore must have outperformed the market (assuming this is all before fees), but not that investors in those other funds outperformed their investments.
2. My example was deliberately the simplest possible example for the sake of readability in a non-technical publication. But in math, when a general proposition is proposed or inference is drawn, if you can find only one counterexample, it invalidates the universality of the proposition. That’s the point of the example. Nevertheless, my example and its means of construction could be considerably elaborated on, with, I’m confident, the same result.
3. I’m glad to hear that Morningstar has modified its language on the “behavior gap.” But I should note that your recent “Mind the Gap 2024” publication does have much the same language on investors underperforming their investments due to buying high and selling low (never mentioning that on the other side of each of their trades, there must be another investor who sells high and buys low), so I don’t know when you changed it. As before, this recent publication leaps to the underperformance inference from the difference between time-weighted and dollar-weighted return, without adequate analysis of the full meaning of that difference.
4. The Appendix to Morningstar’s “Mind the Gap 2024” explains that the annual “Mind the Gap” study “compares funds’ dollar-weighted returns with their time-weighted returns to see how large the gap, or difference, has been over time.” It later says, “investor returns account for all cash flows into and out of the fund to measure how the average investor performed over time.” Yes, but it doesn’t account for what those cash flows were doing between the time they flowed out and the time they flowed in – as does, of course, time-weighted return. That’s the problem. And that’s why the comparison of time-weighted return to IRR is an apples-to-oranges comparison.
Thanks. I'll skip the point by point thing, esp seeing that we're at loggerheads on a few of these. On the last point, we are treating the entire universe of funds in the study (numbering over 25,000) as if they were a single fund - pooling their initial assets, their monthly flows, and their ending assets. We are estimating the return of the average dollar in that mega-fund. I agree that the language in the appendix could more clearly explain this. But if your question is how are we accounting for what happened before an inflow or after an outflow to/from a particular fund (an outflow or an inflow, respectively, from/to another fund), the short answer is we are attempting to capture that by including that other fund in the study as part of this pool. And we will have the data for that other fund, indicating how much it started with, ended with, and the flows in between, all of that data pooled to form that mega-fund I've described. One question for you: Suppose we're talking about one fund. You'd reject the validity of estimating the return of the average dollar invested in that fund because if it differed from the TWR unless we could substantiate the shortfall?
If it’s all one fund but dollars flow out of it sometimes, you don’t know where those dollars are going. They could go into a low- or zero-interest bank account, but it’s also possible they go into something that earns more than the fund itself, and you won’t know that from your measure. However, I will agree that if the fund is the aggregate of all US equity funds, then if money flows out of it, it is likely the money goes into something that earns less than the equity market in the long run. If that’s the case, and if IRR minus time-weighted return registers that shortfall, it’s likely that the shortfall is due to investors not being in the higher-earning equity market all of the time. That is the more supportable inference, not bad market timing.
Jeff, please see this article: https://www.advisorperspectives.com/articles/2025/07/14/trial-barry-ritholtz. It has a simple, elegant refutation of the “mind the gap” methodology that Morningstar uses. (At least I find it persuasive.). Please let me know what you think.
Thanks. W/re to the critique that a 'gap' is a red herring because markets are zero sum (to paraphrase), yes if you were to run this study across all markets as a whole then it should approximate zero. We are focused on the US mutual fund market, a subset of the global market.
W/re to the critique that our estimates of the gap could misrepresent the dollar-weighted return of individual investors - yeah that's true and that's why we have never positioned the study in those terms to my knowledge. For instance, we do not state 'The average investor earned x% while the average fund earned x+1.50%'. We also do not refer to it as a 'behavior gap' as the author has. We are attempting to estimate the return of the average dollar invested in US funds and ETFs. In doing so, we pool the beginning assets, monthly net flows, and ending assets of *all* funds and ETFs that existed at the start of our 10-year study period.
Notwithstanding that, the author seems to imply an investor who temporarily left the hypothetical index fund earned zero return in the interim as far as our study's methodology is concerned. That isn't accurate. We would capture the outflow from that index fund and then if that investor put the proceeds in a different fund that's part of our study universe (which the author's example doesn't imagine, as it's a one-fund universe in that example) we'd capture the inflow to that other fund and then when that investor redeemed from that other fund to return to the index fund we'd capture the outflow from that other fund and the inflow to the index fund, respectively.
But let's suppose our study was that single hypo index fund the author presents and we are trying to estimate the return of the average dollar in that hypo fund. In year one, the two investors put down $20k in aggregate and finished the year with $21k, which they pulled out entirely. In year two, since there was no money in the fund, its return is irrelevant. In year three, the two investors put in $23,100 (which assumes the first investor sat in cash, earning nothing, and the other investor, glass half full, earned 20% in wherever he put the year 1 proceeds, though it's not clear from the example if we're talking about the same monies vs., glass half empty, him plunking down more cash) and earned 5% on that to finish with $24,255. The average dollar in that pool of capital--the index fund--earned a 5% IRR. That's what we're attempting to estimate.
The author might object to us saying that the average dollar in his hypo universe--the one fund--earned 5% despite the fact that the second investor earned substantially more. But that second investor either exited our universe or put additional monies in. We don't know. (In the real world, ie, our study where it's not a single fund rather thousands, it's likely we capture that investor's contribution to the other fund where she made hay and then redeemed to put the enlarged sum to work in the index fund in question.)
Hope that helps.
Jeff
Jeff,
Here are responses to your comments:
1. You say, “…yes if you were to run this study across all markets as a whole then it should approximate zero. We are focused on the US mutual fund market, a subset of the global market.”
ME response: Then are you saying that if you focus on the US mutual fund market, you get a positive gap (i.e., US mutual fund investors underperform their investments), while if you focus on the whole world, you have to get approximately zero? That would imply that when US mutual fund traders compete with other traders, they suffer by comparison – presumably by timing the market badly. Do you believe that, and if this is really evidence for that (which I don’t believe), is there any other corroborating evidence? Why would US mutual fund traders specifically be worsted by those they trade with?
2. You say, We “do not refer to it as a 'behavior gap' as the author has.”
ME response: But surely you are aware that that is exactly how virtually everybody else in the investment field is interpreting your measure. And furthermore, Morningstar’s pronouncements on its web sites very clearly imply that this is what the measure shows. Take this web page for example: https://global.morningstar.com/en-gb/funds/buy-and-hold-or-time-the-market-mind-the-gap. Shouldn’t you enlighten them that they should not interpret it that way? And if you do, shouldn’t it be presented as a correction to your previous pronouncements? And if they should not interpret it that way, what interpretation should they put on it, if any?
3. You say, “the author seems to imply an investor who temporarily left the hypothetical index fund earned zero return in the interim as far as our study's methodology is concerned.”
ME response: No, I do not imply that. I merely assumed that investor A in my example did earn a zero return in the interim while investor B did not. I did not assume that all investors earned a zero return in the interim.
4. You say, “We would capture the outflow from that index fund and then if that investor put the proceeds in a different fund that's part of our study universe … we'd capture the inflow to that other fund and then when that investor redeemed from that other fund to return to the index fund we'd capture the outflow from that other fund and the inflow to the index fund, respectively.”
ME response: This is news to me. To the best of my knowledge your description of the “mind-the-gap” measure does not portray it this way. And anyway, how could you do that unless you knew exactly where that particular investor was placing her funds at all times? And that would only be for one investor. Of course there are individual investors who happen to time the inflows and outflows to their mutual funds wrong, but then there must be others on the other sides of their trades who do it right. But you can’t track the investments of a whole category of investors the way you just described in such a way as to claim (as I’m afraid you do) that a whole group of them exhibits a “behavior gap.”
5. You have two paragraphs beginning with, “But let's suppose our study was that single hypo index fund the author presents…etc.”
ME response: I’m sorry but I don’t fully understand those paragraphs. Perhaps you can clarify further.
Michael
Thanks. Some thoughts which I've numbered to correspond to your responses above:
1. As you know, we lack the data to estimate the return of the average dollar in realms that are less transparent/measurable than the U.S. fund market. It is axiomatic that if markets are zero sum, if we are observing a shortfall to total return in one area (US funds), there would have to be an offsetting surplus elsewhere (ex US funds). As to why that would be so, information asymmetry; scale advantages; transaction frictions; etc. I do not have evidence for this, though I would imagine there have been studies of this in the academic literature. Does that call the possibility of a gap's existence into question? I suppose you could argue that (which you have) but I'm guessing that if we found that, say, the time-weighted pre-fee return of a particular group of funds diverged from that of a representative benchmark, we would not question its validity because we can't empirically find the offset elsewhere?
2. You're correct that in the past we referred to it as a 'behavior gap' and I think others who have done research in this area have used similar terms. However, we have made a concerted effort in recent years to avoid that term because it evokes the notion of people acting rashly, etc. when in reality dollar-weighted outcomes can vary owing to context and other circumstances. A good example is smaller gaps among allocation funds, that likely owing to such funds being heavily used in the 'gilded cage' context of a retirement plan that encourages long-term saving and discourages transacting mainly by automating routine tasks like rebalancing, etc. Conversely, narrower vehicles like sector funds, which are less commonly found in that context, have tended to bedevil investors.
To make this more explicit, page 3 of the study states: "To be sure, inopportunely timed purchases and sales—buying high or selling low on impulse, for instance—can chip away at investor returns. But even laudable practices like investing a portion of every paycheck or regularly rebalancing can open a gap between investor results and reported total returns. Given that nuance, it's not advisable to view this study's findings as a parable of "dumb money" or evidence of individual investors' fallibility."
3. We'll agree to disagree on this point I guess. I think that your example--which I realize is presented for simplicity's sake and that's fine--is misleading in suggesting that the study is so truncated as to omit large swaths of activity. The study encompasses more than 25,000 funds representing the lion's share of the industry's assets. As I mentioned, we pool beginning assets, monthly flows, and ending assets, thereby attempting to capture movements between funds that are a part of our study universe and also including funds that have been merged or liquidated away.
4. We lay out our methodology in the report (it's in the appendix). It has been clearly disclosed for some time now and I'm not familiar with the description you're attributing to us ("To the best of my knowledge your description of the “mind-the-gap” measure does not portray it this way."). We are not attempting to estimate the dollar-weighted return of particular investors. Rather, we are trying to estimate the return of the average dollar in our pool. To avoid creation bias, we limit the study to funds that existed at the start of our study period, including funds that subsequently died.
5. I was imagining your hypo example was the 'US fund industry' and then applying the methodology that we use in conducting our study, whereby we pool the beginning assets, periodic flows, and ending assets so as to derive the IRR. As mentioned, I think it's an oversimplistic construct given that it imagines us being blind to a year's activity (if it's a one-fund universe and both A and B do not capture the return of that one fund in year 2 then it effectively means they went elsewhere and thus the study is blind to 1/3 of the starting investors' activity). But mainly I wanted to explain how the study would approach it methodologically.
fwiw.
Thanks.
Jeff
1. Yes, if the time-weighted pre-fee return of a particular group of funds diverged negatively from that of a representative benchmark (i.e. the total market) then you could say that those funds underperformed the market, but not that the investors in those funds underperformed their investments – only that they chose the wrong ones. And you could conclude that all the other funds therefore must have outperformed the market (assuming this is all before fees), but not that investors in those other funds outperformed their investments.
2. My example was deliberately the simplest possible example for the sake of readability in a non-technical publication. But in math, when a general proposition is proposed or inference is drawn, if you can find only one counterexample, it invalidates the universality of the proposition. That’s the point of the example. Nevertheless, my example and its means of construction could be considerably elaborated on, with, I’m confident, the same result.
3. I’m glad to hear that Morningstar has modified its language on the “behavior gap.” But I should note that your recent “Mind the Gap 2024” publication does have much the same language on investors underperforming their investments due to buying high and selling low (never mentioning that on the other side of each of their trades, there must be another investor who sells high and buys low), so I don’t know when you changed it. As before, this recent publication leaps to the underperformance inference from the difference between time-weighted and dollar-weighted return, without adequate analysis of the full meaning of that difference.
4. The Appendix to Morningstar’s “Mind the Gap 2024” explains that the annual “Mind the Gap” study “compares funds’ dollar-weighted returns with their time-weighted returns to see how large the gap, or difference, has been over time.” It later says, “investor returns account for all cash flows into and out of the fund to measure how the average investor performed over time.” Yes, but it doesn’t account for what those cash flows were doing between the time they flowed out and the time they flowed in – as does, of course, time-weighted return. That’s the problem. And that’s why the comparison of time-weighted return to IRR is an apples-to-oranges comparison.
5. See my response to point 2.
All the best,
Michael
Thanks. I'll skip the point by point thing, esp seeing that we're at loggerheads on a few of these. On the last point, we are treating the entire universe of funds in the study (numbering over 25,000) as if they were a single fund - pooling their initial assets, their monthly flows, and their ending assets. We are estimating the return of the average dollar in that mega-fund. I agree that the language in the appendix could more clearly explain this. But if your question is how are we accounting for what happened before an inflow or after an outflow to/from a particular fund (an outflow or an inflow, respectively, from/to another fund), the short answer is we are attempting to capture that by including that other fund in the study as part of this pool. And we will have the data for that other fund, indicating how much it started with, ended with, and the flows in between, all of that data pooled to form that mega-fund I've described. One question for you: Suppose we're talking about one fund. You'd reject the validity of estimating the return of the average dollar invested in that fund because if it differed from the TWR unless we could substantiate the shortfall?
If it’s all one fund but dollars flow out of it sometimes, you don’t know where those dollars are going. They could go into a low- or zero-interest bank account, but it’s also possible they go into something that earns more than the fund itself, and you won’t know that from your measure. However, I will agree that if the fund is the aggregate of all US equity funds, then if money flows out of it, it is likely the money goes into something that earns less than the equity market in the long run. If that’s the case, and if IRR minus time-weighted return registers that shortfall, it’s likely that the shortfall is due to investors not being in the higher-earning equity market all of the time. That is the more supportable inference, not bad market timing.