Newsletter #167 Home
Newsletter Contributors
Previous Issues
BARRA Home




It's Hard to Distinguish
Skill from Luck

Seven Quantitative
Insights into Active
Management -
A Summary



Forecast Return
Distribution in Aegis
Risk Manager 2.0

The Market
Impact Model™ - Part 3

Automating the
Investment Process



Ron Kahn Looks
Back on 11 Years
at BARRA


The BARRA Brainteaser
for Summer 1998

Solution to the Spring
1998 Brainteaser



A Beautiful Mind

Automating the Investment Process

by Reiner Bohlen

Background

Equity markets have evolved at a remarkable rate over the last 30 years. In such a dynamic environment, opportunities frequently present themselves to practitioners astute enough to spot them. Once spotted, however, opportunities and strategies need to be verified to gain a measure of their prospective value. One popular process for verifying an investment strategy is the backtest, which, while straightforward in theory, can often be tedious and time-consuming in practice. Since competitive advantage belongs to those who can quickly implement ideas and strategies, investment success is often a function of the underlying investment infrastructure, an idea that is explored briefly in this article.

Backtests can take many forms within the constraints of the available technology. Prior to the widespread use of computers, backtesting was in all likelihood a frail and infrequently performed process. One can speculate that backtesting successes were due more to “low-hanging fruit” than methodological rigor. With the advent of optimizer software and advances in Modern Portfolio Theory, our ability to include additional explanatory variables in the backtesting process increased dramatically. Today, advances in both hardware speed and software sophistication have made it possible to test the validity of strategies that are amazingly intricate by historical standards.

That said, the road to a successful backtest remains fraught with both theoretical and practical peril. On the theoretical side, hindsight often allows us to fight yesterday’s battles with great success, only to find that the anticipated strategic advantages have been arbitraged away prior to implementation. Data mining also lurks as a significant danger (see Ron Kahn’s article “Data Mining is Easy” in the Winter ’98 Horizon). On the practical side, an effective backtest might require the use of many resources, including data streams, data vendors, software vendors, application developers, hardware platforms, and personnel to run the process.

Despite its perils, backtesting has become a popular tool in the investment community. Used by money managers and sponsors alike, backtesting not only permits investment strategy verification but also facilitates the development of sophisticated sales and marketing strategies as well. Given the growth in quantitative investment, most practitioners are aware of the theoretical pitfalls of backtesting and can cast a critical eye on the results (e.g., the old saw about halving the information ratio of a backtest before interpreting the results). Thus, the logistical difficulties of performing a backtest loom somewhat more ominously than the theoretical considerations.

Case Study

To illustrate, let’s consider a day in the life of Jane Johnson, a typical portfolio manager at Average Capital Manage-ment—Equity, Inc. (ACME, Inc.). Jane has an intuition about some subtle aspect of market behavior and has developed an investment strategy based on that intuition. She would like to verify her strategy and (should it prove successful) implement it in a new fund ACME is starting. To assess its value, Jane decides to run a backtest. The problem(s)

Over the years, ACME has discovered that technological advances such as increased analytical power, speed, and sophistication often carry the hidden price of nearly unmanageable complexity. This is especially true of systems and processes that have “grown organically” over long periods of time. In a perfect world, all software tools would recognize all other software tools and simply “snap” together like Lego™ blocks. This is unfortunately not the case. With the proliferation of technological tools and software vendors, the office can be an electronic Tower of Babel.

For example, ACME’s backtesting environment includes the following elements:

• Internal policies that constrain potential strategies (in a bound manual)
• The parameters of the strategy itself (on the back of a cocktail napkin)
• Historical alpha information provided by Vendor A (in CSV format)
• Historical asset information, risk model, and optimizer provided by Vendor B (using a proprietary data format and a closed software system that runs under Windows)
• Raw macroeconomic data from Vendor C (in fixed-width format)
• Proprietary software for manipulating alphas and custom data (command-line DOS utilities written by an anonymous analyst in the ’80s)
• Refined custom data (in CSV format, output from the DOS utilities)

First, Jane will need to formulate her strategy in terms of constraints, penalties, and proprietary custom data that the optimizer and associated tools will recognize. Then comes the arduous process of getting the data into shape. This involves finding Vendor A’s alpha information and ACME custom data on the network, massaging the content with the DOS utilities, re-formatting it both manually and with other utilities, and storing the results in a temporary location. Next, Jane will need to run the optimizer, hand-set parameters for the month in question, and start the optimization process. Finally, she’ll take the output portfolio from the optimization, use it as input for the next month, and, as they say in the shampoo business, “Lather, rinse, and repeat. . .” By the time the backtest is complete, hours, days, or even weeks may have passed.

This complexity and tedium constitute a significant barrier to the timely completion of backtesting as well as a variety of other investment tasks. The result is that those tasks are completed more slowly, less frequently, or not at all. There has to be a better way. Doesn’t there?

The solution

Absolutely, and here it is:

Every money manager develops a unique investment process, often over years, incorporating the best technology from each passing era. BARRA has approached the automation problem by opening up its analytical engines to programmatic control and creating a set of easily accessible BARRA functions. With the Aegis Developer’s Toolkit, over 100 of the most commonly used Aegis functions are available in a number of popular programming environments. Specifically, Risk Manager, Optimizer, and Screening Tool functions are available in C, C++, VB, VBA, and other languages. (Performance Analyst functions are planned for next year). Instead of requiring a human pilot to manually manipulate data and optimize every month of a backtest, organizations can create integrated software environments that reflect their unique business processes and technological history.

ACME decides to create a unified backtesting application called Backtest Bench (BB) that integrates each of the elements discussed earlier. The impact of such a toolkit on the day-to-day jobs of people like Jane Johnson, Backtester, is immeasurable:

• BB’s strategy description and development screen has the internal policies embedded in the software or an external table, preventing the user from formulating an inappropriate strategy. It includes start and end dates, relevant variables, constraints, penalties, and all other necessary information used by ACME. It even supports complex strategies whose constraints and penalties are functions of previous results, like adaptive strategies and feedback loops.
• In addition to providing easy access to the features and parameters that ACME does use, BB streamlines the interface by eliminating extraneous features that the firm doesn’t use. By providing sharper focus and removing unnecessary complexity, ACME will find that it can train more employees faster and more effectively than was previously possible.
• Once the parameters of the strategy are specified, BB retrieves the alpha and macroeconomic data from the network. It then calls some of the DOS utilities to manipulate the data (those that supported non-interactive processing), while relying on internal re-implementation of the other utilities. The technological history of the company will dictate the path chosen here (“Is it more cost effective in the long run to reuse or re-implement the DOS utilities?”).
• Rather than using a closed software system that is interactive-only, BB relies on an optimizer implemented as an NT service using Microsoft Messaging Queue. This optimizer service is constantly “alive” on a server, waiting for requests to be added to its queue. Once the optimization is complete, the server passes the results back to the program that called it. This flexible implementation of the optimizer allows other custom applications to connect to it on an ad hoc basis, maximizing reuse and modularity.
• When the user clicks on the “Backtest!” button, magic happens. Gone are the hours, days, and weeks of babysitting the backtesting process. Gone is the tedium of finding and formatting data files by hand, month by endless month. Gone are the agony and heartache associated with the drudgery of the process. Jane Johnson has but to take the resulting optimized portfolios and feed them to a performance attribution tool to evaluate the results. Then she can get on with what she set out to do in the first place—implement her strategy. The focus is shifted from the backtesting process to the backtesting results; from the software domain to the finance domain. Ahhh. . .

This doesn't imply that implementing an automated investment process is trivial. It's not. Every systems integration project comes bundled with its own peculiar set of trials and tribulations. In the case study, ACME had to invest the time and money necessary to examine, specify, and create the new software process that made Jane's job so much easier. Different legacy situations involve different levels of pain ("Can we re-use the old utilities, or do we need to re-implement them?"). Ultimately, each organization needs to do a cost-benefit analysis to chart the appropriate course.

Financial analytical toolkits aren't silver bullets. They do, however, mark the next stage in the evolution of investment process automation and integration. If enhanced productivity and improved ease-of-use lead to faster verification and implementation of new strategies, which in turn result in significant financial gains, it can definitely be worth the pain.