All Data Should Be Treated Like Big Data


The following article is excerpted from a white paper titled “Advanced Analytics to Optimize Capital Deployed” by Gabe Lowy, a former sell-side technology analyst who is founder of TechTonics Advisors, a provider of strategic communication for technology companies.  The link to the white paper is

To improve decision outcomes and operational performance, financial markets firms are incorporating new technologies such as advanced analytics to speed trends analysis and behavior pattern recognition in ever-larger data sets, including big data. The new technologies apply not only to investment risk and return management, but also to product development, customer retention, and compliance.

Tech-Tonics believes that financial markets firms should treat all of their data as big data. Advances in underlying technologies now enable massive data sets of current activity to be ingested and analyzed in real time. Structured and unstructured data can be better integrated to gain historical perspective and more accurately anticipate and predict what will happen next. Advanced statistical, data mining and machine learning algorithms can be applied to these mixed data sets to dig deeper to find patterns faster and more cost-effectively than legacy business intelligence (BI) tools.

Key Technology Advances

Source: Tech-Tonics Advisors

Advanced analytics enable financial markets firms to optimize capital deployed. The key benefit is a deeper understanding of the relationship between portfolio risk/reward models and market liquidity and volatility. These techniques also allow investment firms and their clients to identify and understand the key performance variables (KPVs) that drive portfolio performance.

Analytics can present points of reference to orient intelligent choices, identify mispriced assets and focus decisions with the benefit of underlying data. In this capacity, analytics can provide guidance. But to be most effective, data governance must ensure the highest data quality. If users trust the data they work with models and outcomes will be more reliable.

Since information sets have outgrown the capacities of even the largest teams of modelers to understand interactions and dependencies a unified analytics framework is needed. Such a framework provides competitive advantage by enabling firms to avoid risk in an agile manner and re-allocate capital more quickly in the face of changing risk and return environments.

Addressing Data Management Challenges

For years, financial markets firms have struggled to harvest business intelligence from myriad data silos. IT teams regularly spend up to 25% of their time and budgets on complex and costly integration projects across departmental and organizational silos. The emergence of big data only exacerbates the data management issue.

Despite an insatiable demand for more data, most fund managers cannot consistently and accurately identify the input variables that drive portfolio performance. Yet as costs for sources and data management continue to rise, the return on that “research” continues to decline.

As alpha generation has become fleeting and increasingly expensive it is more important than ever for market participants to understand KPVs and potential outcomes. The need to beneficially anticipate the impact of KPVs and events on capital at risk and liquidity necessitates the implementation of more advanced data integration and analytics tools.

Automated data aggregation, integration and advanced analytics are effecting a fundamental transformation in investment research and portfolio management processes. They exceed an investment professional’s ability to assimilate, correlate and interpret fundamental company or market data at scale. Advanced analytics speeds decision times with objectivity – thereby eliminating bias – to enable more accurate predictive and valuation models. Knowing the KPVs that drive portfolio performance can not only improve competitiveness, but also help avoid catastrophic losses, damaged reputations or fines for non-compliance.

The Case for Unified Analytics

A unified framework that incorporates all of the financial metrics a firm relies on would improve operational efficiency through better decision making. Newer storage and processing technologies coupled with advanced analytics allow for the dynamic monitoring of positions, markets, events, and capital flows within a unified platform. Such a system allows for calculations and dissemination of analyses in real time.

A unified approach also benefits time management and simplifies work flow – acting like a radar system – showing managers what matters, helping them see what they should focus on and what they can ignore, and separating the real news, the market moving news, from the noise. It also provides the tools to better monitor traders and portfolio managers as an early warning system to short-circuit undue risk taking or rogue behavior.

Real-time quantitative analyses of multiple scenarios provide more timely, relevant and accurate views of at-risk capital and potential return across different asset classes. With the ability to instantaneously see exposures, capital reserves, returns and liquidity (both firm and market) across many different scenarios – often at once – risk and compliance management can be strengthened and simplified in a consistent, efficient manner at every level of the organization.

Finally, a byproduct of the increased automation a unified analytics platform provides is that it allows IT to focus on higher priorities, such as application availability and user experience. IT spends less time aggregating and integrating data in preparation for analysis – which often consume up to 80% of time in analytics projects – and more time on driving business value that achieve performance and financial objectives. The day-to-day user experience shifts to applying insight vs. laboriously and repetitively deriving insight. In short, this is the better, modern way to work, with the benefit of technology and automation.


About Author

Gabriel Lowy is the founder of TechTonics Advisors, a strategic communications consultancy for technology companies that bridges vision, strategy, portfolio and markets with customers and investors to build value for all stakeholders. TechTonics synthesizes complex concepts and technologies into clearly understood and actionable value messages. During the previous 15 years, Gabe was a multiple award-winning technology analyst at several Wall Street firms. He began his career with Andersen Consulting. Gabe has published over 1,000 reports and articles on different sectors and companies in the technology industry. Follow Gabe on twitter @GabrielLowy1 and learn more at

Leave A Reply