- The outperformance observed before a typical smart beta index is launched virtually disappears once it’s live, yet most investors are making decisions on backtest results.
- Two traits common to backtests—overfitting (or data-snooping bias) and ignoring transaction costs—bias investors’ live return expectations higher than may be realistic.
- By expecting lower performance than backtest results show, questioning how those results were achieved, and selecting a strategy built on sound economic theory, smart beta investors can frame more realistic performance expectations.
Our headquarters in Newport Beach is only 50 miles from the Hollywood studios, although the drive can take up to two hours in rush-hour traffic. But far more than traffic separates the studios’ world from ours. Film directors and actors are allowed multiple “takes” so each scene we see on the big screen is perfect. Occasionally, however, some productions are taped live, and the unplanned cannot be edited out. Who can forget the 2017 Oscars when La La Land was mistakenly announced as the winner of Best Picture?
Like the producers of the Oscars and other live shows, such as Saturday Night Live,investors don’t have the luxury of re-takes—investing committed capital is “live.” Portfolio results can and will go wildly off script, but there are no do-overs. With smart beta, investors often make decisions using simulations, or backtests. Backtests, like big movie productions, can be subject to editing.
Our research shows that approximately two-thirds of smart-beta-index track records are backtests and that most live track records extend no longer than a decade, which implies that much of the investment outcomes reported by smart beta providers are from simulations. In addition, much of the live history that exists is developed without having substantial assets invested in the strategy. Our collective 22 years on the front lines of smart beta research and investor engagement has shown us that investors nearly always base their decisions on these backward-looking, frictionless results.1 We have no problem with that, if it’s done with eyes wide open. Let’s look at ways investors can set better expectations to maximize the benefits of these strategies.
Backtest vs. Live
A backtest, a frequently used tool to frame forward-looking return expectations, is conveniently easy to calculate2 and can be extremely helpful in proving a solid economic intuition with data—and, of course, with 100% hindsight. A backtest can also be useful in gaining a better understanding of the risks associated with an investment strategy—when the strategy is likely to underperform, and why.
Heavy reliance on backtest results can, however, be a harmful activity if investors are not fully aware of the limitations related to the simulated results. We examine the performance of 125 US equity smart beta indices on which exchanged traded funds (ETFs), characterized as strategic beta by Morningstar, are based. We exclude sector indices; indices for which we are not able to obtain the launch date; and indices with less than one year of backtest or live return data. If two or more ETFs track the same index, we include that index only once. The average live history of our universe of smart beta indices is 7 years, and the average total available history is about 21 years.