The following is a guest article published by Joe Gits, CFA, CEO and co-founder of Context Analytics (formerly Social Market Analytics), a cutting-edge analytics company that converts social media data into actionable intelligence for financial markets customers.
At Context Analytics we are seeing a rise in the deployment of in-house LLMs using Retrieval Augmented Generation (RAG) on cleaned and parsed data. The path to Agentic AI is linking in-house LLMs to algorithms triggering alerts, auto populate reports (analyst reports, credit reports, risk reports, etc.) and even execute some tasks (filing standardized reports, algorithmic trades based on defined triggers).
We are seeing companies bringing much of their current workflow systems in-house as they replace dashboard vendors, a trend driven primarily by cost savings. This Direct-to-the-Underlying-Data approach allows for greater control of data for both security and privacy purposes. This approach allows for integration of internal and external data for a seamless user experience.
Previously, achieving this holy-grail of data integration for better decision making was economically unfeasible for all but the largest companies. Users were left with multiple vendor dashboards they would reference throughout the day. In-house LLMs have radically changed the costs of data integration as well as the insights generated and have become essential productivity gains and expense management.
When designing an in-house AI strategy, companies must:
- Evaluate the strengths and weaknesses of different open-source LLMs;
- Evaluate user workflows and how operations fit together within an organization;
- Evaluate what outside data sources are needed;
- Evaluate data cleaning and parsing services for internal documents;
- Evaluate data permission requirements based on end-user needs;
- Customize LLMs for the different internal applications;
- Link action-based algorithms to LLMs (Agentic AI).
Given the difficulties in hiring and retaining AI/ML developers, consultants will play a critical role in guiding companies as they build their in-house LLMs and Agentic AI solutions.
Data quality is critical to an in-house approach. Garbage-In, Garbage Out remains true. Three key attributes to look for in a data provider are Cost, Quality & Accuracy, and Available History. When evaluating the cost of an external data provider, consider if the data is cleaned and parsed. When evaluating a data cleaning/parsing service for internal documents, the most common benchmark is the fully loaded cost to hire staff internally. Again, given the difficulty in hiring and retaining experienced AI/ML developers, it is rare that an internal process is less expensive than an outsourced provider.
Quality and Accuracy is hard to achieve when using an LLM with unstructured and non-parsed documents. We typically see maximums of 80% accuracy with first time runs of data, accuracy improves significantly with each training run. Industry expertise and proven track records for data quality are critical considerations.
Finally, a RAG approach with historical data in addition to real-time updates is important for providing the LLM with context needed to provide meaningful insights and accurate results for Agentic AI purposes.
Our view is different than the mainstream that see AI companies extracting high SaaS rents from clients with outsourced AI solutions. While these drop-in solutions are easy to implement and have lower upfront costs, the problems with this approach include:
- Data that remains siloed with different specialized AI solution providers (i.e. different providers for finance, management and manufacturing), limiting cross-functional insights and productivity gains from Agentic AI applications;
- Data security and privacy concerns when trying to implement AI solutions on internal data.;
- Lack of flexibility;
- Additional monthly costs for each outsourced AI provider.
The team at Context Analytics are excited about the implementation of LLM solutions for finance and see exponential leaps in insight and overall productivity of investment professionals.
Contact Info.
Joe Gits, CFA
CEO
Context Analytics, Inc.
Office: (312) 788-2625
Cell: (630) 640-4368
Email: joeg@contextanalytics-ai.com
www.contextanalytics-ai.com
901 W Jackson Blvd # 204, Chicago, IL 60607