Big Data and Investment Research: Part 2


Last week we wrote about why buy-side firms are considering adding “big data” analytic techniques to their research process.  This week, we will investigate the impact that “big data” is now having on the sell-side and alternative research industry.

The “Big Data” Evolution

As we mentioned last week, the term “big data” when applied to investment research is the practice of regularly collecting, storing and analyzing huge amounts of structured and unstructured data sources to generate predictive insights with the sole purpose of generating consistent investment returns.

Despite pronouncements to the contrary, this is not really a new phenomenon in the research business as most of the best sell-side and alternative research firms have been doing just this for many years — albeit to a more limited extent.  Countless research analysts have conducted formal channel checks, surveys, or shopping bag counts; or they have licensed unique commercial data feeds; they have warehoused this data; and they have analyzed these unique data points as part of their research process for many years.  In fact, some research franchises like Midwest Research and it’s many spinoffs were widely known for this type of data-driven research process.

However, in recent years as computing power has grown, computer storage costs have fallen, and the internet has exponentially increased the availability of both structured and unstructured data, both the capability and interest in expanding the data-driven research model also increased with buy-side research consumers and the third-party research producers that serve them.

History of “Big Data” Investment Research

As mentioned in last week’s article, one of the first third-party “big data” firms to serve institutional investors was Majestic Research, founded in 2002, who collected, warehoused, and analyzed a number of unique data sets to identify investable signals for asset managers.  However, Majestic Research found this business model was difficult to maintain as quantitatively oriented hedge funds that initially used the service felt the predictive edge of this analysis deteriorated as more investors used it.  In other words, Majestic Research could not scale its research business.

In response, the firm decided to hire fundamental securities analysts who could leverage their proprietary data and statistical analysis to produce data-driven fundamental research.  They found the market for this type of research was much broader than pure quantitative investors, and these buy-side clients were less worried that the data was too widely distributed.  They valued the insight Majestic provided more than the underlying data which led to that insight.  As discussed last week, Majestic was acquired by ITG in 2010 and this data-driven research product became the foundation for ITG’s research product since.

However, other firms saw that the Majestic model could enhance the value of the traditional fundamental research product.  The largest single “big data” research initiative was rolled out by Morgan Stanley in 2008 with its Alpha Wise initiative.  Initially, AlphaWise conducted and charged hedge funds and asset managers for customized primary research, including market research, web bots, and expert surveys.  However, eventually, AlphaWise morphed into a unique data-driven research product (they call it evidence based) that Morgan Stanley clients could access based on hundreds of terabytes of proprietary survey and web scraped data.

Then in 2009, San Francisco-based alternative research firm, DISCERN started up to build a “big data” research model to meet the varied needs of different types of institutional investors.  As mentioned last week, DISCERN is an institutional equity research firm which covers a wide range of sectors, including specialty pharmaceuticals, financial services, energy, and real estate investment trusts.  Its research is based on the statistical analysis of huge amounts of structured and unstructured data to identify unique investment signals, and then overlays this data analysis with contextual insights generated by experienced sell-side analysts to identify key trends, market deflection points, and other potential investable opportunities.  Discern provides a wide range of unbundled services, including allowing buy-side clients to license its data, the predictive signals it has developed, or the extensive research it generates.

Consequences for the Research Industry

So, does the adoption of “big data” by these three research firms have any long-term consequence for the research industry?  We think so.  Clearly, these three firms have found extensive market acceptance of their data centric research products.  Based on our own analysis, DISCERN has been one of the faster growing alternative research firms in the industry over the past few years.  In addition, research produced by the AlphaWise team has been some of the most highly demanded research produced by Morgan Stanley.

If investment research firms want to compete with growing research firms like Majestic (now ITG), Morgan Stanley’s AlphaWise, or DISCERN, they are going to have to make significant changes to their traditional research businesses.  They will need to invest in building (or renting) scalable data warehouses and other technical infrastructure.  In addition, they will need to start thinking about what public data they want to collect; what commercial data they want to license; and more importantly, what proprietary data they should create by conducting longitudinal surveys, implementing web scraping technology, or utilizing other primary research techniques.  They will also need to hire data scientists or analysts skilled in building predictive analytics to work alongside their traditional analysts with deep company or industry expertise.

In addition, they will need to think through their business model and product set.  Do they only want to sell access to their analysts and the research reports they wrote as they did in the past?  Or, do they want to unbundle their research process and allow customers to cherry pick exactly what they want, including data, investment signals, full research reports, custom work, etc.

And of course, how do they want their clients to pay for this research – through hard dollar subscriptions, CSAs, trading, or some combination of the above? It is interesting to note that currently, all three of the companies mentioned earlier in this article, ITG, Morgan Stanley and DISCERN enable their clients to pay for these “big data” research services by trading through their trading desks.

Winners & Losers in “Big Data” Research?

Given the significant financial investment required in people, data, and technology, we suspect the obvious winners in rolling out “big data” research services are likely to be sell-side investment banks.  Clearly, many of these firms have produced data driven research in the past based on the analysts they have hired.  The move to a “big data” focus will really be a commitment on the part of the firm to basing all their investment research on a deep statistical analysis of underlying data sources.

However, it is important to note that many sell-side firms will not choose to make the switch to a “big data” based research process, nor will everyone that tries to do so succeed.  One of the major impediments to success is a bank’s unwillingness to make the financial commitment necessary to succeed.  Certainly, the downturn in equity commissions over the past five years and the struggle to make equity research pay could convince many management teams that an investment in “big data” research is just too risky with no obvious payoff.   Another reason some firms will fail in its big data efforts is their inability to adapt the firm’s culture to this new approach to investment research.

So, can alternative research firms succeed in developing “big data” research services?  We think so, though we do not think it will be a quick road to success.  In the past, both Majestic Research and DISCERN were alternative research firms that became successful in this space, though in each case, the firms developed their coverage and their product offering incrementally rather than trying to roll out broad coverage at the outset.

Similarly, we suspect that other alternative research firms will be successful in the future by initially focusing on a limited number of sectors based on a discreet set of proprietary data they have collected and few predictive signals they have developed.  These firms will be able to expand their businesses as they gain commercial traction by adding new sectors, more proprietary data, and additional analytic signals.

Another possible winner in the “big data” research space could be existing data aggregators or predictive analytics providers who decide to move up the value chain with their data products by adding context and investment insight to their offerings by hiring investment research analysts.  Unfortunately, we think that very few data aggregators or analytics providers will take the risk of stepping outside their domain expertise to enter the investment research business.  Consequently, we  don’t expect to see too many of these types of firms enter the research business.


In our view, the acceptance of “big data” techniques in the investment research industry is a forgone conclusion.  Buy-side investors have increasingly exhibited their appetite for unique datasets and for data driven research over the past decade.  To meet this hunger, a number of sell side and alternative research firms have well developed efforts under way, and we suspect that a few significant new entrants are likely to announce initiatives in 2014.

The real question remaining is how existing research firms plan to respond to this trend.  As we mentioned last week, the adoption of “big data” techniques enables a firm to develop a proprietary edge – whether it is through the collection of unique datasets or the development of proprietary signals that have predictive value.  We believe that as more research firms adopt a “big data” oriented research process, it will be increasingly harder for other traditional research firms, with no discernible proprietary edge to compete.  The days where a research firm could succeed solely on the basis of having a cadre of analysts with deep industry experience might be over.



About Author

1 Comment

  1. The other leg of this story is not about proprietary insights but about using big data to get more conventional analysis (analysis of the data we already have) quicker, cheaper and more accurate. Practitioners know that a lot of time and effort go into making the data amenable to the tools available to us, which are not a good fit for the problems we’re trying to solve. A new set of tools opens up new approaches and better solutions to long-standing problems around attribution, tail-risk contribution, asset-liability management, style analysis and contribution to expected shortfall. Those are all analytics that are important to responsible investing but are poorly served by the existing relational-database conventional tools.

Leave A Reply