Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating the Lab for newer labs #3

Merged
merged 5 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 48 additions & 21 deletions Instructions/Lab_Instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Login to your VM with the following credentials...
1. [Compare Results of RAG responses using Vector search, Reranker or GraphRAG](#compare-results-of-rag-responses-using-vector-search-reranker-or-graphrag)

# Part 0 - Log into Azure
Login to Azure Portal with the following credentials.
Open Edge in the lab environment and login to Azure Portal with the following credentials.

1. Go to [Azure portal](https://portal.azure.com/) `https://portal.azure.com/`
- Username: [email protected](User1).Username+++
Expand All @@ -50,6 +50,8 @@ Login to Azure Portal with the following credentials.
Also pick *No storage account required* and select your subscription.
![Screenshot of the Azure toolbar with the Cloud Shell icon highlighted by a red box.](./instructions276019/select_storage.png)

Click Apply after selecting your subscription

3. At the Cloud Shell prompt, enter the following to clone the GitHub repo containing exercise resources:

> +++git clone https://github.com/Azure-Samples/mslearn-pg-ai.git+++
Expand Down Expand Up @@ -228,24 +230,19 @@ The <code spellcheck="false">azure_openai</code> schema provides the ability to

The docs will shows the two overloads of the <code spellcheck="false">azure_openai.create_embeddings()</code> function, allowing you to review the differences between the two versions of the function and the types they return.

2. To provide a simplified example of using the function, run the following query, which creates a vector embedding for the <code spellcheck="false">opinion</code> field in the <code spellcheck="false">cases</code> table. The <code spellcheck="false">deployment_name</code> parameter in the function is set to <code spellcheck="false">embedding</code>, which is the name of the deployment of the <code spellcheck="false">text-embedding-3-small</code> model in your Azure OpenAI service:
2. To provide a simplified example of using the function, run the following query, which creates a vector embedding for a sample query. The <code spellcheck="false">deployment_name</code> parameter in the function is set to <code spellcheck="false">embedding</code>, which is the name of the deployment of the <code spellcheck="false">text-embedding-3-small</code> model in your Azure OpenAI service:

```sql
SELECT
  id,
  name,
  azure_openai.create_embeddings('text-embedding-3-small', LEFT(opinion, 8000)) AS vector
FROM cases
LIMIT 1;
SELECT azure_openai.create_embeddings('text-embedding-3-small', 'Sample text for PostgreSQL Lab') AS vector;
```

the output looks similar to this:


```sql-nocopy
id |      name       |              vector
----+-------------------------------+------------------------------------------------------------
507122 | Berschauer/Phillips Construction Co. v. Seattle School District No. 1 | {0.020068742,0.00022734122,0.0018286322,-0.0064167166,...}
id |   vector
----+-----------------------------------------------------------
| {0.020068742,0.00022734122,0.0018286322,-0.0064167166,...}
```


Expand Down Expand Up @@ -437,18 +434,17 @@ We already created a sample Legal Cases RAG application so you can explore RAG a

1. In [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) is using the results from vector search to answer your questions. Try any query to test the limits of the application.

**Suggestions for queries:**
**Suggested for query:**
1. `Water leaking into the apartment from the floor above. What are the prominent legal precedents from cases in Washington on this problem?`
2. `When the landlord is sued in court for leaking pipes, infer and give examples of the number of times there was a favorable decision for the lessee?`

### Review Accuracy of vector search queries:

For the 2 sample question, we have manually identify 10 legal cases that will produce the best answers. To explore the accuracy of vector search follow the instruction below:
For the sample question, we have manually identifed 10 legal cases that will produce the best answers. To explore the accuracy of vector search follow the instruction below:

1. Click the graph icon in the chat bubble to see with cases were used to answer the question.
![Graph screenshot](./instructions282962/RAG-app-demo-graph-icon.png)

2. From the Citation Graph, you will see Vector search only retrieve 40% of the most revelvant cases. The orange indicates what was retrieved to answer the questions, and green indicates what should be retrieved for the sample question.
2. From the Citation Graph, you will see Vector search **only retrieve 40% of the most relevant cases**. The orange indicates what was retrieved to answer the questions, and green indicates what should be retrieved for the sample question.
![Recall of Graph screenshot](./instructions282962/RAG-app-demo-recall-graph.png)

===
Expand All @@ -462,9 +458,21 @@ Read more about reranking in [our blog post](https://aka.ms/semantic-ranker-solu

![Semantic Reranker image](./instructions282962/semantic-ranking-solution-postgres-on-azure.png)

### Using a Reranker
>[!alert] Make sure you are using **pgAdmin** for the following steps.
### Understanding improved accuracy of semantic reranker:

Using the same example from the vector search example in the preview section. To explore the accuracy of semantic reranker follow the instruction below:

1. Use the [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/)

1. Select Semantic Ranker from the top bar and try the sample query from the previous example.
![select Reranker](instructions282962/selectReranker.png)

1. Click the graph icon in the chat bubble to see with cases were used to answer the question.

1. From the Citation Graph, you will see semantic reranker has a slighty improve accuracy, and **retrieves 60% of the most revelvant cases**.

### How to implement a reranker for queries
>[!alert] Make sure you are using **pgAdmin** for the following steps.

1. Before we execute the reranker query to improve the relevance of your search results. We should understand the following important snippet of code for reranking.
> [!alert]
Expand Down Expand Up @@ -502,7 +510,8 @@ Read more about reranking in [our blog post](https://aka.ms/semantic-ranker-solu
```

1. Run the query.
>[!tip] **This query is going to take around 3 secs**

>[!tip] This query is going to take around 3 seconds


you will get a result like this:
Expand All @@ -522,14 +531,30 @@ you will get a result like this:
558730 | Burns v. Dufresne
```

===

### What is GraphRAG
GraphRAG uses knowledge graphs to provide substantial improvements in question-and-answer performance when reasoning about complex information. A Knowledge Graph is a structured representation of information that captures relationships between entities in a graph format. It is used to integrate, manage, and query data from diverse sources, providing a unified view of interconnected data. [Apache Graph Extension](https://age.apache.org/age-manual/master/index.html) (AGE) is a PostgreSQL extension developed under the Apache Incubator project. AGE is designed to provide graph database functionality, enabling users to store and query graph data efficiently within PostgreSQL.

Read more about Graph RAG in [our blog post](https://aka.ms/graphrag-legal-solution-accelerator-pg-blog).

![graphrag-postgres-architecture.png](instructions282962/graphrag-postgres-architecture.png)

### Using a GraphRAG

### Understanding improved accuracy of GraphRAG:

Using the same example from the vector search example in the preview section. To explore the accuracy of semantic reranker follow the instruction below:

1. Use the [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/)

1. Select GraphRAG from the top bar and try the sample query from the previous example.
![select GraphRAG](instructions282962/selectGraphRAG.png)

1. Click the graph icon in the chat bubble to see with cases were used to answer the question.

1. From the Citation Graph, you will see semantic reranker has a slighted improve accuracy, and **retrieves 70% of the most revelvant cases**.

### How to implement graph queries for GraphRAG
>[!alert] Make sure you are using **pgAdmin** for the following steps.

1. Before we execute the graph query to improve the relevance of your search results. We should understand the following important snippet of code for reranking
Expand Down Expand Up @@ -570,13 +595,13 @@ Read more about Graph RAG in [our blog post](https://aka.ms/graphrag-legal-solut
![Open file in pgAdmin](./instructions276019/open-file.png)

1. Now to run the *graph query* to create the node and connections between cases in your database.
>[!tip] This query is going to take around 5 secs
>[!tip] This query is going to take around 5 seconds

1. Create new query tool on the same connection

![Creating new query tool](./instructions276019/new-query-tool.png)

1. Click Open File icon, and find the reranker_query in the Downloads folder. `/Downloads/mslearn-pg-ai/Setup/SQLScript/graph_query.sql`
1. Click Open File icon, and find the graph_query in the Downloads folder. `/Downloads/mslearn-pg-ai/Setup/SQLScript/graph_query.sql`

![Open file in pgAdmin](./instructions276019/open-file.png)

Expand All @@ -598,6 +623,8 @@ you will get a result *like* this:

```

===

## Compare Results of RAG responses using Vector search, Reranker or GraphRAG

1. In [RAG application](https://abeomorogbe-graphra-ca.gentledune-632d42cd.eastus2.azurecontainerapps.io/) select the 'Vector Seach' option in the top bar. Try the sample query
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 0 additions & 21 deletions Setup/Infra/deploy.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ param adminLoginPassword string
@description('Unique name for the Azure OpenAI service.')
param azureOpenAIServiceName string = 'oai-learn-${resourceGroup().location}-${uniqueString(resourceGroup().id)}'

@description('Unique name for the Azure AI Language service account.')
param languageServiceName string = 'lang-learn-${resourceGroup().location}-${uniqueString(resourceGroup().id)}'

@description('Restore the service instead of creating a new instance. This is useful if you previously soft-delted the service and want to restore it. If you are restoring a service, set this to true. Otherwise, leave this as false.')
param restore bool = false

Expand Down Expand Up @@ -134,28 +131,10 @@ resource azureOpenAIEmbeddingDeployment 'Microsoft.CognitiveServices/accounts/de
}
}

@description('Creates an Azure AI Language service account.')
resource languageService 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
name: languageServiceName
location: location
kind: 'TextAnalytics'
sku: {
name: 'S'
}
properties: {
customSubDomainName: languageServiceName
publicNetworkAccess: 'Enabled'
restore: restore
}
}

output serverFqdn string = postgreSQLFlexibleServer.properties.fullyQualifiedDomainName
output serverName string = postgreSQLFlexibleServer.name
output databaseName string = casesDatabase.name

output azureOpenAIServiceName string = azureOpenAIService.name
output azureOpenAIEndpoint string = azureOpenAIService.properties.endpoint
output azureOpenAIEmbeddingDeploymentName string = azureOpenAIEmbeddingDeployment.name

output languageServiceName string = languageService.name
output languageServiceEndpoint string = languageService.properties.endpoint