Where We Are in Generative AI Compliance – Part 1


Generative AI is transforming investment management and data analysis, making compliance an essential area of focus for both investors and regulators. In November 2023 we covered the topic as part of a workshop titled Legal & Compliance Considerations for Data, AI, and Large Language Models in Finance where we were joined by Sanaea Daruwalla of Zyte and Jessica Margolies of Schulte Roth & Zabel.

Following on from this session due to topic popularity, in January 2024, a panel was organized at our Next Level conference which was held in New York on the 18th of January. The panel, “Compliance Considerations for Generative AI for Investment Managers and Data Providers,” examined AI’s role in the financial services vertical and the related regulatory landscape.

For more information on Eagle Alpha events and to register your interest to attend our upcoming Alternative Data Conference in London on May 16th, please click here.

Expert Views Shared By:

  • Jessica Margolies, Special Counsel at Schulte Roth & Zabel: Jessica specializes in non-traditional research methods, including the use of generative AI.
  • Emilie Abate, Director at Iron Road Partners: With extensive experience at the SEC and now at Iron Road Partners, Emilie has a comprehensive view of AI and data from a regulatory standpoint.
  • Alik Sokolov, CEO of Sibli: Alik’s background in AI consulting and venture capital, along with his academic research in financial machine learning, positions him to understand the practical applications of AI in finance. His work focuses on helping investment firms incorporate AI into their research processes.

Disclaimer: The views and opinions expressed by the panellists are their own and do not necessarily reflect the official policy or position of the organization hosting the event, its affiliates, or any other agency, organization, employer, or company. Statements made by the panellists are intended for discussion purposes only and are not intended as investment advice.

How Generative AI is Being Approached by Funds

This section focuses on the strategic approaches that funds are adopting to harness the potential of Generative AI, with insights from industry experts on risk management, regulatory compliance, and the evolving use cases within the investment environment.

Alik: “Generative AI is really a paradigm for training machine learning models in a certain way that allows them to generate data for us… we’ve recently been able to scale up these models to be much larger than we ever did in the past. And in doing so we realised that they had a lot of interesting emerging capabilities that give them a lot of utility for business use.”

Key Takeaways:

  • Alik stressed that understanding the risks involves considering two main factors: the nature of the training data (whether it’s internal or from external providers like Azure, AWS, or OpenAI) and how the model is being utilized, particularly given its propensity for random errors.
  • Jessica built on this by categorizing generative AI use into two main types: internal and external. She pointed out the different risks and control needs associated with each type. Key considerations include understanding the scope of use, the nature of data input (such as IP, PII, or confidential information), and contractual aspects, especially when using licensed data from alternative data vendors.
  • Emilie highlighted the SEC’s capabilities in cybersecurity and the potential regulatory repercussions of a breach, highlighting the importance of cyber diligence when onboarding AI vendors. The SEC, according to Emilie, views cybersecurity breaches not only as incidents but also as compliance failures, particularly in protecting PII.
  • Jessica shared insights on updating diligence processes considering generative AI’s nuances. This includes broader questions about the scope of use, data input and output, intellectual property rights, confidentiality concerns, and bias in the AI models. She emphasized the evolving nature of these processes, requiring ongoing collaboration between different stakeholders, including investment firms, vendors, and regulatory experts.

Practical Applications and Compliance Challenges

This section provides expert insights on balancing the innovative uses of generative AI with the importance of robust compliance and risk management strategies.

Jessica: “The way we think about it is there’s sort of two buckets… internal LLM generative AI use case that’s hosted on a client… and the external public facing enterprise version… What you’re inputting to each one of those types of tools is subjected to different kinds of risks and needs different kinds of controls.”

Emilie: “Compliance really needs to be in the mix with the business side of things and figure out how these tools are being used… making sure that whatever outputs are coming out… to put a disclaimer on it right, that says, ‘This was generated by AI and has not been verified for accuracy.’”

Key Takeaways:

  • Alik addressed the challenges faced by larger enterprises in transitioning generative AI applications from experimentation to production. He explained that while it’s relatively easy to achieve 95% accuracy with generative AI, attaining higher accuracy levels requires significantly more effort and resources. Alik emphasized the need for rigorous testing and understanding of these models before fully integrating them into operational processes, especially given the high-risk tolerances of large organizations.
  • Emilie discussed the increasing use of tools like ChatGPT in investment research and the compliance implications that arise as these tools become more integral to the investment decision-making process. She advised that compliance departments should work closely with investment teams to understand how these AI tools are being used. Emilie suggested best practices like saving AI-generated outputs and attaching disclaimers about their accuracy to mitigate regulatory risks.
  • Drawing parallels with the evolution of alternative data, Jessica highlighted how the investment sector is adopting a more forward-thinking and collaborative approach towards integrating generative AI. She noted that the policies and procedures being developed are broader and more fluid, to accommodate the rapid changes and unknowns inherent in this technology. Jessica stressed the importance of understanding the application of generative AI and adapting compliance policies accordingly.

For more information on Eagle Alpha events and to register your interest to attend our upcoming Alternative Data Conference in London on May 16th, please click here.


About Author

Mike Mayhew is one of the leading experts on the investment research industry. In addition to founding Integrity Research, Mike is on the board of directors of Investorside Research Association, the non-profit trade association for the independent research industry, and a frequent speaker on research industry trends and developments. Mike has over thirty years of research industry experience. Email: Michael.Mayhew@integrity-research.com

Leave A Reply