• The market for asset managers is inefficient, with significant information asymmetries and incentive problems
  • We can’t directly observe manager quality; we can only estimate it using noisy signals
  • We can improve our estimates by investing in better due diligence technologies
  • The value of more accurate due diligence is very high, since most active managers destroy value


In pursuit of the objective of better outcomes from active management, it’s helpful to step back and reflect on some fundamental questions about the nature of the manager search and due diligence process.

Portfolio Manager

A Portfolio Manager awaits an interview with a fund selection committee.

In this post, we sketch out some of the economic foundations that underlie the Empirically Compass, which is the framework that guides our approach to investment strategy evaluation and selection. These foundations inform our framing of the problem and the method of its solution: what are the challenges that are preventing investors from getting better outcomes from active management, and how can we help address them?

In the interest of staying focused on the key issues, we won’t present a full model in this post – complete with agents, choices and objective functions – but we do have one, and we’d be happy to discuss it with you; just get in touch.

The Market for Asset Managers

Active management raises both general equilibrium and partial equilibrium questions. General equilibrium questions include:

  • How efficient are markets? Does alpha exist? What are the sources of excess returns?
  • What proportion of managers are skilled?
  • Why do unskilled managers exist and persist? Why don’t they go out of business, or – better yet – why do they go into business in the first place?
  • What are the incentive problems created by delegated asset management, why do they arise, and how might they be solved?
  • How are each of these effects trending over time?

While these topics are primarily of interest to academics and perhaps policy makers, they’re also relevant for investors. If there is no alpha in an asset class, for example, then it’s futile to pay an active manager to try to generate it. Indeed, implicit in the decision to hire an active manager are a number of beliefs, including that (1) alpha exists, (2) skilled managers exist, (3) it’s possible to identify them ex ante, and (4) the costs of doing so (such as search and monitoring) outweigh the expected benefits.

Basic Requirements to Select an Active Strategy

Diagram of requirements

Partial equilibrium questions on the other hand are less about the system as a whole, and more about the choices faced by an individual market participant. For example, even if mutual fund managers destroy value in aggregate, it could well still make sense for a given investor to buy mutual funds – if she has a way of picking the right ones.

What’s already evident is that there are two different markets at work here: the market for assets, and the market for (active) asset management. These are interlinked:

  • The more efficiently investors can identify skilled managers, the more money will be allocated to active management. The more difficult or costly it is to find skilled managers, the more money will be allocated to passive alternatives.
  • The less efficient the market for assets, the more opportunities for asset managers to add value. But the more effort being put into capitalizing on inefficiencies, and the more money allocated to active managers, the more efficient the asset markets become.

The costs of due diligence don’t have a direct relationship to assets under management, but the potential gain from identifying good managers does, as the higher returns are multiplied by a larger asset base. Therefore, large investors have stronger incentives to invest in sourcing good managers than small ones, who might be better served by staying passive and sacrificing potentially higher returns in exchange for cost savings.

Quality and Noise

Putting aside the reasons why for a moment, the empirical fact is that there’s some sizeable fraction of asset managers which lack an edge (informational advantage), or otherwise don’t add value net of their fees. Academics sometimes call them “noise traders”, or simply unskilled.

At Empirically, we refer to these managers as low-quality. We believe that quality is a multi-dimensional concept which is partially subjective, since each investor has a unique preference and opportunity set. Additionally, there are multiple ingredients to a good manager beyond informational advantage, which is at once both too narrow and too vague to capture overall quality.

Almost nobody would knowingly hire a low-quality manager, so why do low-quality managers attract any clients? In fact, not only do they exist, they are actually the majority of managers. Clearly, there are major inefficiencies in the market for active managers: due to flawed diligence processes, many investors end up mistaking luck for skill, or talk for results. To parallel the term for fund managers, we could call these investors “noise allocators”, since they’re making decisions based on uninformative signals. At Empirically, our mission is to make sure that our clients are in the successful minority of informed allocators.

The challenge is that quality isn’t directly observable. Indeed, many managers themselves may not even know their own true quality (though they still know more than outsiders). While some unskilled opportunists may start funds to take advantage of this setup, the majority of active managers likely genuinely believe that they’re of high quality.

Of course, if fund managers turn out to be overconfident in their own abilities, it’s not so bad for them: while investors have underperformed, they’ve still earned handsome fees. Indeed, the costs of bad performance borne by the fund manager may even be zero, if a new set of investors view his learnings and experience of failure as an asset while marketing his next fund!

Where does quality come from? Quality accrues from three levels, each of which may contribute (or detract) to quality to differing degrees for a given product, depending on the nature of the investment strategy:

  • The Portfolio Manager might be most important in a concentrated long-only fund or a global macro hedge fund, as the primary idea generator and decision maker.
  • The Investment Team might be most important in a systematic strategy or private equity fund, where different contributors play complementary roles.
  • The Firm might be most important in a high-frequency trading strategy, where access to technology and resources is the primary driver of competitiveness.

Broadly, there are two ingredients:

  • Exogenous Quality (Skill): Factors outside of the manager’s control during the period of the relationship.  This depends on things like education, prior experience, natural aptitudes, etc.
  • Endogenous Quality (Effort): Factors the manager can influence over the period of the relationship. This depends on things like incentive structure, work ethic, resourcing decisions, etc.

Unfortunately, this all gets even more complicated. Both components of quality – exogenous skill and endogenous effort – can increase and decrease over time, at each of the three levels, as can their contributions to outcomes. Examples of common changes that can alter quality include learning (i.e. from experience), hires/departures from a team, and changes in market structure that render particular capabilities and skills more or less effective. Ultimately, what fund investors care about most is quality over their own (future) investment horizon.

Quality Can Vary Over Time at Every Level

Quality Can Vary Over Time at Every Level

Fund Manager Due Diligence: Measuring Quality

Watch the Story: Quality Distributions

Given the complexities and risks inherent in active management, it’s perhaps surprising that it remains so widespread – although passive and quasi-passive approaches have been gaining share in recent years. Active fund investors clearly believe that the benefits will outweigh the risks; most are aware of the statistics, but believe that they possess the fund selection ability needed to overcome the odds. Observed outcomes show that this is not always true.

In this discussion, we’re focusing on investment due diligence, and leaving out a discussion of operational due diligence. While operational diligence is equally critical, it’s usually more straightforward: professional fund selectors and consultants are able to audit and measure operational quality with comparatively high accuracy. Our work at Empirically views operational excellence as “table stakes”: managers which lack it need to be screened out for compliance and fiduciary reasons. Where we can lend our expertise is the question of discerning quality differences among seemingly similar managers, who all meet operational requirements.

Since quality can’t be directly observed, we have to produce an imperfect estimate of it by collecting signals that we believe are correlated with it. The big challenge is with fund evaluation is that these signals can be very noisy (unreliable), due to the tremendous component that random variation – or luck – can have on outcomes, even over a relatively long period. That’s one reason why past performance isn’t indicative of future results, even if quality were stable instead of time-varying, and even if the benchmark being used is appropriate: a high-quality manager might end up with mediocre performance, and a low-quality manager might end up with excellent performance. The shorter the measurement period, the more likely this is to occur – but long track records have their own problems.

Types of Due Diligence Metrics

Types of Due Diligence Metrics

Since commonly-used outcome-based measures are so noisy, there’s often a heavy focus on process-based due measures, including qualitative due diligence. But it’s important to keep in mind that process evaluation is also subject to noise: an investment process can be very difficult to observe and judge from the outside.

Two particular issues arise in process-based evaluation:

  • Issue 1: We don’t really know what process results in high quality. Sustainable alpha is by definition a secret sauce; if it had a known recipe, everyone would do it. And then the alpha would disappear…
  • Issue 2: Due to Issue 1, fund managers have a good argument for being secretive. This can make it difficult to monitor and assess the process – especially for hedge funds. Many of the most successful managers disclose little to nothing about the inner workings of their process. If a manager is happy to “pull back the curtain” and reveal everything, which all seems simple and logical enough, should that really be taken as indicating the possession of an alpha-generating strategy?

After collecting a number metrics are collected, they are aggregated to produce an overall ranking or rating, often in the format of a scorecard. The idea is that by combining multiple signals, a more precise estimate of quality can be obtained. That’s true in principle, but the million-dollar (billions-of-dollars?) question is which metrics to use, and how to weight them.

Problem: There Are Infinite Potential Measures and Weights

Manager evaluation metrics

We can come up with an infinite number of candidate metrics; academics and practitioners haven’t reached infinity yet, but there’s certainly no shortage of new ratios and measurement criteria. We can also calculate each metric over many different potential time periods, and then combine them in an infinite number of ways: adding, subtracting, dividing, and weighing.

All of these signals are not created equally. Good due diligence metrics provide unique information about manager quality; however, there are also many “bad” metrics, in the sense of being uninformative or misleading. Basing a decision (partially) off of bad metrics may dilute the strength of the signal about true quality (increase estimation error), or worse, yield a wrong signal (introduce bias).

Finding the Right Selection Process

Bringing this discussion back to its beginning, we see that the search for good asset managers closely resembles those managers’ search for good assets. Just as portfolio managers have many different strategies to screen for and research stocks, bonds, and derivatives, asset allocators may also use many different possible diligence processes to select funds.

With such a vast number of potential processes, it’s clear that there can be a large dispersion between good and bad processes – which translates into a large dispersion in outcomes from delegated management. It’s equally clear that a systematic method is needed to figure out the optimal set of metrics that are predictive of the dimensions of quality, and can be observed objectively with reasonable cost and effort:

  • Bad Process 1: Look at all the metrics, think about them and make an “expert judgment”.
    • Problem: There is too much complexity (infinite metrics) and too much at stake to rely on this approach. Additionally, it may be deemed inadequate by future regulatory or legal scrutiny.
  • Bad Process 2: Pick some metrics, then pick some weights for them, average them together into a “scorecard”, choose the top-scoring manager.
    • Problem: Without a rigorous framework for optimally selecting both the metrics and the weights, the resulting scores are likely to be sub-optimal at best and outright wrong at worst.
  • Empirically Approach: Apply cutting-edge statistical techniques to a large, proprietary data set of fund characteristics to estimate objective predictive models, which are validated out-of-sample and have known accuracy and reliability parameters.

The Value of Better Due Diligence

Institutional investors have strong incentives to invest in predictive analytics and implement better decision frameworks, which reduce the probability that they will mistakenly hire low-quality managers:

  • The performance differential between top-quartile and bottom-quartile funds is significant, and continues to compound over the years of a manager relationship.
  • Bad managers ultimately need to be terminated, and switching costs are high in the form of search costs, portfolio transition, and staff time.
  • Underperformance hurts beneficiaries and invites adverse scrutiny, which can result in career risk, and even litigation risk in a growing number of cases.
  • The larger the portfolio, the larger the dollar value of all of these items.

With this in mind, we invite you to schedule a demo if you‘d like to learn more about how we can augment your diligence efforts and help you identify high-quality external managers.

Furthermore, even the best analytics aren’t enough: as already noted, high quality entails more than “just” possessing sustainable skill at generating alpha relative to an appropriate benchmark. Suppose we have robust, effective predictive analytics for multiple relevant dimensions of quality. The final step is then to harness these analytics to make optimal recommendations or decisions which have the highest likelihood of delivering positive outcomes for all relevant stakeholders, whose preferences may differ from one another. This is where decision exploration and analysis tools can be brought to bear: by providing a structured framework for evaluating alternatives and weighing tradeoffs, these tools ensure that the analytics and other available information are being used to make choices in the most beneficial way.

Author Information: Jordan Boslego is a Partner at Empirically.

Updated September 2020.