LLM Retrieve-Augmented Generation in Finance using MATLAB

Information retrieval involves finding documents in a document collection that contain information relevant to the query. This example will demostrate how to use MATLAB Text Analytic Toolbox™ to generate answers from large language models (LLMs) with reliable information contained in a source document stored in a vector database.

To run this example, it requires

• Statistics and Machine Learning Toolbox™
• Text Analytics Toolbox™
• Database Toolbox™

Load and Preprocess Text Data

Load the Federal Open Market Committee (FOMC) meeting documents where monetary policy decisions are made. FOMC is in PDF format. Extract the text from the PDF file. In this example, we used only one meeting notes for simplicity. Text data can be extensive and often contains significant noise, which can negatively impact statistical and downstream tasks. To utilize the Preprocess Text Data Live Task to clean text data. The world cloud below illustrates word frequency analysis performed on a preprocessed version of the FOMC meeting notes.

Vectorization

Load the pretrained document embedding using the documentEmbedding function. This model requires the Text Analytics Toolbox™ Model for all-MiniLM-L6-v2 Network or all-MiniLM-L12-v2 Network support package. If the support package is not installed, it can be downloaded from Add-Ons Menu.

modelName = "all-MiniLM-L12-v2";
embeddingModel = documentEmbedding(Model = modelName);

Set up the query

Define the query, then retrieve and filter the relevant documents based on the query.

query = "Will Federal Fund rate decrease after FOMC meeting in the following 3 months?"
embeddedQuery = embed(embeddingModel,query)
sqlEmbeddedQuery = "'[" + join(string(embeddedQuery),",") + "]'";
sqlCosineSimilarity = "SELECT text FROM embeddings_FOMC ORDER BY 1 - (embedding <=> " + sqlEmbeddedQuery + ") DESC LIMIT 30";
selectedDocs = fetch(conn,sqlCosineSimilarity);
head(selectedDocs,10)

LLM model Selection and Generate answers

In this section, you will install Large Language Models (LLMs) with MATLAB from Add-Ons Explorer. Define the prompt for the chatbot and generate a response. Select an appropriate LLM, then combine relevant information and the user query to construct a well-formed prompt. Finally, use the model to generate responses based on the inputs.

Reference

1. Large Language Models with MATLAB