Newsletter #167
Home
Newsletter Contributors
Previous Issues
BARRA Home
It's Hard to Distinguish Skill from Luck
Seven Quantitative Insights into Active Management - A Summary
Forecast Return
Distribution in Aegis
Risk Manager 2.0
The Market Impact Model™ - Part 3
Automating the Investment Process
Ron Kahn Looks
Back on 11 Years
at BARRA
The BARRA Brainteaser
for Summer 1998
Solution to the Spring
1998 Brainteaser
A Beautiful Mind
Seven Quantitative Insights into Active Management - Part 7
It's Hard to Distinguish Skill from LuckEarly in this series (Part 2) we learned that information ratios (IR) determine value added: they are the key statistic capturing manager performance. Here we will look at our ability to accurately measure information ratios.
To understand the measurement issues involved, we will need to analyze the standard error of an observed information ratio (IR). The standard error will help us to determine the statistical significance of the observed IR.
We start by writing the IR as:
(1)
where we measure 
and 
, the mean and standard deviation of active or residual returns, over time periods 
, and the in EQUATION 1 annualizes the IR. For example, we might calculate and using monthly data, and the annualization multiplies the monthly ratio by 
.
Because we will assume that the statistical uncertainty in a dominates the statistical uncertainty in the IR,1 let’s rewrite EQUATION 1 as:
(2)
Then, the standard error of the IR is:
(3)
(4)
where N measures the number of observations underlying our measurements of and we have used the classic result for the standard error of an estimated mean.
But we can simplify EQUATION 4 to find the result:
(5)
where T measures the number of years of observation. Surprisingly, this standard error is independent of frequency: Given five years of data, the standard error of the IR is the same whether we use quarterly, monthly, or daily data.
Given the simple result in EQUATION 5, what can we say about the difficulty of measuring information ratios? We have seen that a top quartile manager has an IR of 0.5. How long must we observe such a manager to measure the IR with 95% confidence? We want the t-statistic, the ratio of the IR to its standard error, to exceed 2:
(6)
or
T > 16 (7)
According to EQUATION 7, we require 16 years of data for that level of statistical confidence.2
But requiring so long a time series is quite problematic. Most active managers today probably do not have such lengthy track records. More importantly, managers and strategies may not retain their information ratios over such long periods. Superior information and great opportunities don’t last forever.
So we find ourselves in a situation of some uncertainty, as captured in FIGURE 1. Here we classify managers along two dimensions—skill and luck: The “Blessed” possess both skill and luck. Their businesses thrive, and deservedly so. The “Doomed” lack skill and luck. We quickly and appropriately weed them out.
The other two categories are more problematic. The “Forlorn” possess skill but not luck. Their performance numbers do not convey their true level of skill, and they suffer for that. The final category is the “Insufferable.” They possess luck but no skill, and they thrive. Most managers I know can quickly cite examples of other managers they feel fall into this category.
The real insight at the end of this analysis is that successful active management depends on both skill and luck. Depending on your perspective, this can be good or bad news.
Figure 1 - Distinguishing skill from luck: Good and bad news
1 This is typically true unless the IR is extremely large (above 1.0) and/or the number of observations is very small. Accounting for the uncertainty in
will only increase the standard error, so the analysis here presents the best case.2 Some purists may argue that we should invoke a one-sided test: What is the likelihood that random data (IR = 0) would generate such a large positive IR? We then require only t > 1.65. This one-sided test still demands 11 years of data.
