diff --git a/Instructions/Lab_Instructions.md b/Instructions/Lab_Instructions.md index c374d03..63e77ff 100644 --- a/Instructions/Lab_Instructions.md +++ b/Instructions/Lab_Instructions.md @@ -27,7 +27,7 @@ Login to your VM with the following credentials... 1. [Compare Results of RAG responses using Vector search, Reranker or GraphRAG](#compare-results-of-rag-responses-using-vector-search-reranker-or-graphrag) # Part 0 - Log into Azure -Login to Azure Portal with the following credentials. +Open Edge in the lab environment and login to Azure Portal with the following credentials. 1. Go to [Azure portal](https://portal.azure.com/) `https://portal.azure.com/` - Username: +++@lab.CloudPortalCredential(User1).Username+++ @@ -50,6 +50,8 @@ Login to Azure Portal with the following credentials. Also pick *No storage account required* and select your subscription. ![Screenshot of the Azure toolbar with the Cloud Shell icon highlighted by a red box.](./instructions276019/select_storage.png) + Click Apply after selecting your subscription + 3. At the Cloud Shell prompt, enter the following to clone the GitHub repo containing exercise resources: > +++git clone https://github.com/Azure-Samples/mslearn-pg-ai.git+++ @@ -228,24 +230,19 @@ The azure_openai schema provides the ability to The docs will shows the two overloads of the azure_openai.create_embeddings() function, allowing you to review the differences between the two versions of the function and the types they return. -2. To provide a simplified example of using the function, run the following query, which creates a vector embedding for the opinion field in the cases table. The deployment_name parameter in the function is set to embedding, which is the name of the deployment of the text-embedding-3-small model in your Azure OpenAI service: +2. To provide a simplified example of using the function, run the following query, which creates a vector embedding for a sample query. The deployment_name parameter in the function is set to embedding, which is the name of the deployment of the text-embedding-3-small model in your Azure OpenAI service: ```sql - SELECT -   id, -   name, -   azure_openai.create_embeddings('text-embedding-3-small', LEFT(opinion, 8000)) AS vector - FROM cases - LIMIT 1; + SELECT azure_openai.create_embeddings('text-embedding-3-small', 'Sample text for PostgreSQL Lab') AS vector; ``` the output looks similar to this: ```sql-nocopy - id |      name       |              vector -----+-------------------------------+------------------------------------------------------------ - 507122 | Berschauer/Phillips Construction Co. v. Seattle School District No. 1 | {0.020068742,0.00022734122,0.0018286322,-0.0064167166,...} + id |   vector +----+----------------------------------------------------------- + 1 | {0.020068742,0.00022734122,0.0018286322,-0.0064167166,...} ``` @@ -437,18 +434,17 @@ We already created a sample Legal Cases RAG application so you can explore RAG a 1. In [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) is using the results from vector search to answer your questions. Try any query to test the limits of the application. -**Suggestions for queries:** +**Suggested for query:** 1. `Water leaking into the apartment from the floor above. What are the prominent legal precedents from cases in Washington on this problem?` -2. `When the landlord is sued in court for leaking pipes, infer and give examples of the number of times there was a favorable decision for the lessee?` ### Review Accuracy of vector search queries: -For the 2 sample question, we have manually identify 10 legal cases that will produce the best answers. To explore the accuracy of vector search follow the instruction below: +For the sample question, we have manually identifed 10 legal cases that will produce the best answers. To explore the accuracy of vector search follow the instruction below: 1. Click the graph icon in the chat bubble to see with cases were used to answer the question. ![Graph screenshot](./instructions282962/RAG-app-demo-graph-icon.png) -2. From the Citation Graph, you will see Vector search only retrieve 40% of the most revelvant cases. The orange indicates what was retrieved to answer the questions, and green indicates what should be retrieved for the sample question. +2. From the Citation Graph, you will see Vector search **only retrieve 40% of the most relevant cases**. The orange indicates what was retrieved to answer the questions, and green indicates what should be retrieved for the sample question. ![Recall of Graph screenshot](./instructions282962/RAG-app-demo-recall-graph.png) === @@ -462,9 +458,21 @@ Read more about reranking in [our blog post](https://aka.ms/semantic-ranker-solu ![Semantic Reranker image](./instructions282962/semantic-ranking-solution-postgres-on-azure.png) -### Using a Reranker ->[!alert] Make sure you are using **pgAdmin** for the following steps. +### Understanding improved accuracy of semantic reranker: + +Using the same example from the vector search example in the preview section. To explore the accuracy of semantic reranker follow the instruction below: + +1. Use the [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) +1. Select Semantic Ranker from the top bar and try the sample query from the previous example. +![select Reranker](instructions282962/selectReranker.png) + +1. Click the graph icon in the chat bubble to see with cases were used to answer the question. + +1. From the Citation Graph, you will see semantic reranker has a slighty improve accuracy, and **retrieves 60% of the most revelvant cases**. + +### How to implement a reranker for queries +>[!alert] Make sure you are using **pgAdmin** for the following steps. 1. Before we execute the reranker query to improve the relevance of your search results. We should understand the following important snippet of code for reranking. > [!alert] @@ -502,7 +510,8 @@ Read more about reranking in [our blog post](https://aka.ms/semantic-ranker-solu ``` 1. Run the query. - >[!tip] **This query is going to take around 3 secs** + + >[!tip] This query is going to take around 3 seconds you will get a result like this: @@ -522,6 +531,8 @@ you will get a result like this: 558730 | Burns v. Dufresne ``` +=== + ### What is GraphRAG GraphRAG uses knowledge graphs to provide substantial improvements in question-and-answer performance when reasoning about complex information. A Knowledge Graph is a structured representation of information that captures relationships between entities in a graph format. It is used to integrate, manage, and query data from diverse sources, providing a unified view of interconnected data. [Apache Graph Extension](https://age.apache.org/age-manual/master/index.html) (AGE) is a PostgreSQL extension developed under the Apache Incubator project. AGE is designed to provide graph database functionality, enabling users to store and query graph data efficiently within PostgreSQL. @@ -529,7 +540,21 @@ Read more about Graph RAG in [our blog post](https://aka.ms/graphrag-legal-solut ![graphrag-postgres-architecture.png](instructions282962/graphrag-postgres-architecture.png) -### Using a GraphRAG + +### Understanding improved accuracy of GraphRAG: + +Using the same example from the vector search example in the preview section. To explore the accuracy of semantic reranker follow the instruction below: + +1. Use the [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) + +1. Select GraphRAG from the top bar and try the sample query from the previous example. +![select GraphRAG](instructions282962/selectGraphRAG.png) + +1. Click the graph icon in the chat bubble to see with cases were used to answer the question. + +1. From the Citation Graph, you will see semantic reranker has a slighted improve accuracy, and **retrieves 70% of the most revelvant cases**. + +### How to implement graph queries for GraphRAG >[!alert] Make sure you are using **pgAdmin** for the following steps. 1. Before we execute the graph query to improve the relevance of your search results. We should understand the following important snippet of code for reranking @@ -570,13 +595,13 @@ Read more about Graph RAG in [our blog post](https://aka.ms/graphrag-legal-solut ![Open file in pgAdmin](./instructions276019/open-file.png) 1. Now to run the *graph query* to create the node and connections between cases in your database. - >[!tip] This query is going to take around 5 secs + >[!tip] This query is going to take around 5 seconds 1. Create new query tool on the same connection ![Creating new query tool](./instructions276019/new-query-tool.png) -1. Click Open File icon, and find the reranker_query in the Downloads folder. `/Downloads/mslearn-pg-ai/Setup/SQLScript/graph_query.sql` +1. Click Open File icon, and find the graph_query in the Downloads folder. `/Downloads/mslearn-pg-ai/Setup/SQLScript/graph_query.sql` ![Open file in pgAdmin](./instructions276019/open-file.png) @@ -598,6 +623,8 @@ you will get a result *like* this: ``` +=== + ## Compare Results of RAG responses using Vector search, Reranker or GraphRAG 1. In [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) select the 'Vector Seach' option in the top bar. Try the sample query diff --git a/Instructions/instructions282962/selectGraphRAG.png b/Instructions/instructions282962/selectGraphRAG.png new file mode 100644 index 0000000..c6eecfd Binary files /dev/null and b/Instructions/instructions282962/selectGraphRAG.png differ diff --git a/Instructions/instructions282962/selectReranker.png b/Instructions/instructions282962/selectReranker.png new file mode 100644 index 0000000..14028d3 Binary files /dev/null and b/Instructions/instructions282962/selectReranker.png differ diff --git a/Setup/Infra/deploy.bicep b/Setup/Infra/deploy.bicep index 23e4964..d9c768f 100644 --- a/Setup/Infra/deploy.bicep +++ b/Setup/Infra/deploy.bicep @@ -19,9 +19,6 @@ param adminLoginPassword string @description('Unique name for the Azure OpenAI service.') param azureOpenAIServiceName string = 'oai-learn-${resourceGroup().location}-${uniqueString(resourceGroup().id)}' -@description('Unique name for the Azure AI Language service account.') -param languageServiceName string = 'lang-learn-${resourceGroup().location}-${uniqueString(resourceGroup().id)}' - @description('Restore the service instead of creating a new instance. This is useful if you previously soft-delted the service and want to restore it. If you are restoring a service, set this to true. Otherwise, leave this as false.') param restore bool = false @@ -134,21 +131,6 @@ resource azureOpenAIEmbeddingDeployment 'Microsoft.CognitiveServices/accounts/de } } -@description('Creates an Azure AI Language service account.') -resource languageService 'Microsoft.CognitiveServices/accounts@2023-05-01' = { - name: languageServiceName - location: location - kind: 'TextAnalytics' - sku: { - name: 'S' - } - properties: { - customSubDomainName: languageServiceName - publicNetworkAccess: 'Enabled' - restore: restore - } -} - output serverFqdn string = postgreSQLFlexibleServer.properties.fullyQualifiedDomainName output serverName string = postgreSQLFlexibleServer.name output databaseName string = casesDatabase.name @@ -156,6 +138,3 @@ output databaseName string = casesDatabase.name output azureOpenAIServiceName string = azureOpenAIService.name output azureOpenAIEndpoint string = azureOpenAIService.properties.endpoint output azureOpenAIEmbeddingDeploymentName string = azureOpenAIEmbeddingDeployment.name - -output languageServiceName string = languageService.name -output languageServiceEndpoint string = languageService.properties.endpoint