More Problems With Performance


New York – Recently, we blogged about the difficulty of measuring “good performance” via synthetic returns. This discussion focused primarily on ways in which wildly inaccurate positive forecasts may lead to very high synthetic returns while very close estimates may produce poor synthetic returns. A commenter pointed out further concerns relating to the liquidity of securities held, investor sentiment, and how one would need to discount the near-term effects on performance of superstar analyst recommendations (the “Blodget-Grubman” effect).

In the past, we have weighted the raw performance of investment recommendations by the firm’s “batting average”, in order to give consistent recommendations a higher grade than those who were merely lucky. The ability of an analyst or research provider to recommend “buys” that outperform their “holds” and “sells” is another useful indicator of internal consistency, though the scarcity of “sell” recommendations gives us a relatively small sample size to compare against “buys” and “holds”.

There are further difficulties with measuring performance which might make any simple performance measurement less relevant to an investor. To put it simply: in order to be broadly comparable, the synthetic performance of investment recommendations would have to be calculated net of transaction costs and with due consideration given to volatility.

Transaction costs are, perhaps, the single biggest reason why “real world” returns rarely match synthetic returns. Many of the highest-performing research providers are quantitative shops which may make recommendations on thousands of companies every year. When we track the synthetic performance of such recommendations, common sense suggests that we should discount more actively traded investment strategies against longer term “buy and hold” strategies, due to the higher transaction costs associated with active strategies. Transaction costs are also significantly higher for smaller, thinly-traded securities. There are research firms that track such real world portfolio returns, and we believe that this would be a much more useful input than synthetic returns calculated without transaction costs.

Furthermore, most people are risk-averse. This is why investors demand a risk-premium – all else being equal, people tend to prefer less volatile securities. While it is extremely difficult to quantify risk, there are widely-used measures of systematic risk and risk/return ratios. In addition, we must also take into account liquidity risk – small, thinly-traded securities might not look very volatile if we merely follow market prices in calculating our synthetic return, but an institutional investor following an active trading strategy will quickly run into significant risk due to the illiquid market for such securities. When tracking a research provider’s synthetic performance, we should discount gains by the implied cost of the risk that would be undertaken by following their recommendations in the real world.

At the end of the day, Integrity’s conversations with portfolio managers reveal that many of them pay little attention to a research provider’s stock-picking performance as measured by synthetic returns, being more interested, for example, in thematic ideas or access to management. The issues discussed here and in our earlier blog on the topic help to explain why the real world returns that a portfolio manager is seeking often have little to do with a research provider’s synthetic returns.


About Author

Leave A Reply