|
| 1 | + |
| 2 | +# Module 1: Using Azure Portal to Create Your Index - No Code Required |
| 3 | + |
| 4 | +This module uses the Azure Portal to create your first Azure Search index without writing any code. Following these steps you will: ingest a set of files (clinical-trials); extract both structured and unstructured text from those files; index their content and learn how to query your new index. Finally, we'll use the Azure Portal to project enriched data into a Knowledge Store (new Preview capability), which we'll explore in greater detail in Module 6. |
| 5 | + |
| 6 | +The instructions that follow assume that you have completed all of the pre-requisites found in the [ReadMe](./README.md) to this lab and have provisioned all of the necessary resources to your Azure subscription. If you have not already completed these steps, you will need to do so prior to moving forward. |
| 7 | + |
| 8 | +## Using the Portal Import Data Flow: |
| 9 | + |
| 10 | + |
| 11 | +1. Navigate to your search service, and click the **Import Data** button. This will launch the Import Data Wizard which will guide you through the steps necessary to ingest your data, enrich it and create a search indexer. |
| 12 | + |
| 13 | +  |
| 14 | + |
| 15 | +1. As part of the Import Data Wizard, in the **Connect to your data** tab, you can enter the information necessary to connect to your data source. |
| 16 | + |
| 17 | ++ In the drop down for **Data Source**, choose *Azure Blob Storage*. |
| 18 | + |
| 19 | ++ **Name** your data source *clinical-trials-small*. |
| 20 | + |
| 21 | ++ Set **Data to Extract** to *Content and Metadata* |
| 22 | + |
| 23 | ++ For the **Connection String** click *Choose an existing connection* and select your storage account. Select the *clinical-trials-small* container. |
| 24 | + |
| 25 | + + If you're not able to find the storage account you want to use by selecting *Choose an existing connection* you can always manually add the connection string. To get your connection string, view your storage account in the Azure Portal, select *Access keys* and copy the *Connection String*. Paste this as the *Connection string*. Then add the *Container name* which will be *clinical-trials-small*. |
| 26 | + |
| 27 | + Your screen should now look similar to this: |
| 28 | + |
| 29 | +  |
| 30 | + |
| 31 | ++ Now click **Next** to apply cognitive skills to your data. |
| 32 | + |
| 33 | +## Skillset |
| 34 | + |
| 35 | +In Azure Search, we call extraction and enrichment steps cognitive skills, which are combined into a skillset referenced during indexing. In this exercise, you will be learning how to use the [built-in skills](https://docs.microsoft.com/en-us/azure/search/cognitive-search-predefined-skills) through the Azure Portal. In a later module, we will show you how to attach these skills programmatically and how to build your own [custom skills](https://docs.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-interface). |
| 36 | + |
| 37 | +In the next three steps, you will be working through the three drop-down arrows presented: |
| 38 | + |
| 39 | + |
| 40 | + |
| 41 | + |
| 42 | +### Attach the Cognitive Services |
| 43 | + |
| 44 | +This is the resource you created earlier as part of your intial lab set up and is used to power your pre-built AI models. |
| 45 | + |
| 46 | + |
| 47 | + |
| 48 | +### Add enrichments |
| 49 | + |
| 50 | +Name your skillset: *clinical-trials-small* |
| 51 | + |
| 52 | ++ Make sure to select the **OCR enrichment** to extract **merged_content** field. |
| 53 | + |
| 54 | ++ Now we can apply an enrichment to the merged_content field to extract the locations. |
| 55 | +Change the name of the generated field to be **locations** with a lowercase 'l'. For consistency’s sake, let’s leave the field name locations. |
| 56 | + |
| 57 | ++ Leave all of the other enrichment boxes blank at this time as we will add in additional skills later in the lab. |
| 58 | + |
| 59 | +  |
| 60 | + |
| 61 | + |
| 62 | +### Save enrichments to a knowledge store (Preview) |
| 63 | +As you recall from the introductory session, the knowledge store is a new capability that we introduced into Public Preview in May. Using the Knowledge Store enables you to use your data in scenarios that do not lend themselves naturally to search. Once your data has been loaded into the Knowledge Store, you can do things like kick off RPA, run analytics or visualize in tools like PowerBI. |
| 64 | + |
| 65 | +Projections are your mechanism for structuring data in a knowledge store. For example, through projections, you can choose whether output is saved as a single blob or a collection of related tables. An easy way to view knowledge store contents is through the built-in Storage Explorer for Azure storage. |
| 66 | + |
| 67 | +The knowledge store supports two types of projections: |
| 68 | + |
| 69 | + + Tables: For data that is best represented as rows and columns, table projections allow you to define a schematized shape or projection in Table storage. |
| 70 | + |
| 71 | + + Objects: When you need a JSON representation of your data and enrichments, object projections are saved as blobs. |
| 72 | + |
| 73 | +For this case, we are going to use Azure table projections |
| 74 | + |
| 75 | + |
| 76 | + |
| 77 | +We're going to go ahead and create the Knowledge Store now through the Azure Portal and will come back to the visualizations in a later module. |
| 78 | + |
| 79 | +1. Click choose an existing connection and select your storage account. |
| 80 | +2. Click on **+ Container** to create a new container called *clinical-trials-small-ks*. |
| 81 | +3. **Select** the container created in the above step. |
| 82 | +4. Under **Azure table projections**, make sure *Documents* and *Entities* have been selected. |
| 83 | +2. Click **Next: Customize the target index**. |
| 84 | + |
| 85 | + |
| 86 | +## Index Definition |
| 87 | +In this step, you are designing your Azure Search index. This is an important and powerful part of the index build process as you select the types of Analyzer(s) you want to use and make determinations on features such as which fields and data will be retrievable, filterable, sortable, and searchable. |
| 88 | + |
| 89 | +1. Give your index a name like *clinical-trials-small* |
| 90 | + |
| 91 | +2. Leave **Key** as the default option |
| 92 | + |
| 93 | +3. Under **Suggester name** add sg and set **Search mode** to *analyzingInfixMatching* |
| 94 | + |
| 95 | +4. In the index definition fields: |
| 96 | + + Make sure all the fields are **retrievable**. |
| 97 | + + Make sure that the locations field is **retrievable / facetable / filterable / searchable**. |
| 98 | + + Set **English-Microsoft** as the *Analyzer* for all searchable fields since the content is in English. |
| 99 | + + Select **Suggester** for trials, metadata_author, metadata_title and locations |
| 100 | + + You can make layoutText not searchable/retrievable since we won’t need it for this workshop. |
| 101 | + |
| 102 | +  |
| 103 | + |
| 104 | + 5. Click on **Next: Create an indexer**. |
| 105 | + |
| 106 | +## Indexer Definition |
| 107 | + |
| 108 | +1. Name the indexer *clinical-trials-small* . |
| 109 | +2. Set the **Schedule** to Once |
| 110 | +3. Click on the **Advanced options** drop down and note that that the index key is Base-64 encoded by default. |
| 111 | + |
| 112 | +  |
| 113 | + |
| 114 | +4. Click on **Submit**. Then wait 2 or 3 minutes or so for the indexing to occur – then go check the status of your indexer on the portal. |
| 115 | + |
| 116 | +  |
| 117 | + |
| 118 | + |
| 119 | +  |
| 120 | + |
| 121 | +## Searching the Content |
| 122 | +Now that the content has been indexed, we can use the portal to test some search queries. Open the **Search explorer** and enter a search query such as "MPS" to allow us to find all document that refer to the disease MPS, and press "Search". Try adjusting the query with different phrases and terms to get an idea of the content. |
| 123 | + |
| 124 | +  |
| 125 | + |
| 126 | +Let's try a few additional queries: |
| 127 | + |
| 128 | +Search for references to "Gaucher's" disease and do hit highlighting of the content where there is a reference to it: |
| 129 | +``` |
| 130 | +gauchers&highlight=content |
| 131 | +``` |
| 132 | +Notice as you scroll through the results that the English-Microsoft Analyzer was able to pick up variations to this phrase such as "Gaucher" and "Gaucher's" and highlights them using default <em> </em> tags. |
| 133 | + |
| 134 | +Add a parameter &$count=true to determine that there are 8 documents that refer to "Gaucher's" disease: |
| 135 | +``` |
| 136 | +gauchers&highlight=content&$count=true |
| 137 | +``` |
| 138 | + |
| 139 | +### Searching the Content using Postman |
| 140 | + |
| 141 | +The search explorer is useful for performing queries like this, however most developers want to use external tools to start working against the service. For that reason, open Postman to perform the rest of the below search queries. To set up the queries we will set the Headers as: |
| 142 | +* api-key: [Enter Admin API Key from Azure Search portal] |
| 143 | +* Content-Type: application/json |
| 144 | + |
| 145 | +You can retrieve the API key by pulling up your search service in the Azure Portal, selecting Keys, then copying one of the available admin keys. |
| 146 | + |
| 147 | +When we configured the Indexing of the content, we asked for locations to be extracted from the content. Let's take a look at this by searching for morquio disease and limiting the results to only return the metadata_title, locations fields. Remember to update {name of your service} with the name of your search service. |
| 148 | +``` |
| 149 | +GET https://{name of your service}.search.windows.net/indexes/clinical-trials-small/docs?api-version=2019-05-06&search=morquio&$select=metadata_title,locations |
| 150 | +``` |
| 151 | + |
| 152 | +Here's what the request will look like in Postman: |
| 153 | + |
| 154 | +  |
| 155 | + |
| 156 | +Notice how the *locations* field is a Collection (or array of strings) that includes all the Location names extracted from the content. |
| 157 | + |
| 158 | +Let's try to group all of the *locations* by using Faceting. |
| 159 | +``` |
| 160 | +GET https://{name of your service}.search.windows.net/indexes/clinical-trials-small/docs?api-version=2019-05-06&search=morquio&$select=metadata_title,locations&facet=locations |
| 161 | +``` |
| 162 | +We can see how the search results has added a list of the top locations and how often they are found in documents that talk about Morquio. |
| 163 | + |
| 164 | +Next, let's filter the results to documents that refer to Morquio and have a Location of "Emory University" |
| 165 | +``` |
| 166 | +GET https://{name of your service}.search.windows.net/indexes/clinical-trials-small/docs?api-version=2019-05-06&search=morquio&$select=metadata_title,locations&$filter=locations/any(location: location eq 'Emory University') |
| 167 | +``` |
| 168 | + |
| 169 | +As a final query, we will use the autocomplete capability to suggest terms that match what a user types. You have likely seen this in search boxes where users start typing and the system quickly suggests potential matches. Notice how this request is a POST as opposed to a GET. |
| 170 | + |
| 171 | +``` |
| 172 | +POST https://{name of your service}.search.windows.net/indexes/clinical-trials-small/docs/autocomplete?api-version=2019-05-06 |
| 173 | +``` |
| 174 | +Set the body of the request as "raw" and include: |
| 175 | +```json |
| 176 | +{ |
| 177 | + "fuzzy": true, |
| 178 | + "search": "hea", |
| 179 | + "suggesterName": "sg", |
| 180 | + "autocompleteMode": "twoTerms" |
| 181 | +} |
| 182 | +``` |
| 183 | +Notice how this request uses a suggesterName called "sg". You will recall that when you configured the index, you selected some columns to be used to power these autocomplete requests. |
| 184 | + |
| 185 | + |
| 186 | +### Next: [Module 2: Visualizing the Results with a Demo FrontEnd](Module 2.md) |
0 commit comments