Skip to content

Commit 47a9f2b

Browse files
committed
Merge remote-tracking branch 'origin/main' into vNext-Dev
2 parents 5579436 + 79af91d commit 47a9f2b

18 files changed

+69
-60
lines changed

Makefile

+4-4
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ build-containers: extract-env
2424
infrastructure: check-subscription ## Deploy infrastructure
2525
@./scripts/inf-create.sh
2626

27-
extract-env: extract-env-debug-webapp extract-env-debug-functions ## Extract infrastructure.env file from TF output
27+
extract-env: extract-env-debug-webapp extract-env-debug-functions ## Extract infrastructure.env file from Terraform output
2828
@./scripts/json-to-env.sh < inf_output.json > ./scripts/environments/infrastructure.env
2929

3030
deploy-webapp: extract-env ## Deploys the web app code to Azure App Service
@@ -39,10 +39,10 @@ deploy-enrichments: extract-env ## Deploys the web app code to Azure App Service
3939
deploy-search-indexes: extract-env ## Deploy search indexes
4040
@./scripts/deploy-search-indexes.sh
4141

42-
extract-env-debug-webapp: ## Extract infrastructure.debug.env file from TF output
42+
extract-env-debug-webapp: ## Extract infrastructure.debug.env file from Terraform output
4343
@./scripts/json-to-env.webapp.debug.sh < inf_output.json > ./scripts/environments/infrastructure.debug.env
4444

45-
extract-env-debug-functions: ## Extract local.settings.json to debug functions from TF output
45+
extract-env-debug-functions: ## Extract local.settings.json to debug functions from Terraform output
4646
@./scripts/json-to-env.function.debug.sh < inf_output.json > ./functions/local.settings.json
4747

4848
# Utils (used by other Makefile rules)
@@ -87,4 +87,4 @@ run-data-migration: ## Run the data migration moving data from one resource grou
8787
python ./scripts/extract-content.py
8888

8989
manual-inf-destroy: ## A command triggered by a user to destroy a resource group, associated resources, and related Entra items
90-
@./scripts/inf-manual-destroy.sh
90+
@./scripts/inf-manual-destroy.sh

docs/deployment/autoscale_sku.md

+16-34
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,6 @@ You may find better settings to fit your needs. This document explains how this
99

1010
The Azure Functions Service Plan Autoscale settings are defined in the file located at `/infra/core/host/functions/functions.tf`. These settings enable automatic scaling of the Azure Functions Service Plan based on CPU usage metrics.
1111

12-
13-
1412
**File Location:** `/infra/core/host/functions/functions.tf`
1513

1614
#### Scaling Rules
@@ -29,7 +27,6 @@ The Azure Functions Service Plan Autoscale settings are defined in the file loca
2927
- **Time Window:** `5 minutes`
3028
- **Scaling Action:** Decrease capacity by `2` with a cooldown of `2 minutes`.
3129

32-
3330
## App Service Plan Autoscale for Enrichment App
3431

3532
### Overview
@@ -58,50 +55,41 @@ The App Service Plan Autoscale settings for the enrichment app are defined in th
5855

5956
### Customization
6057

61-
To customize the App Service Plan Autoscale settings, modify the parameters mentioned above in the specified Terraform file. And Run the `make infrastructure` command.
62-
63-
58+
To customize the App Service Plan Autoscale settings, modify the parameters mentioned above in the specified terraform files. And Run the `make infrastructure` command.
6459

6560
# SKU Settings Documentation
6661

6762
### Overview
6863

69-
The SKU settings for all Service Plans are defined in the file located at `/infra/main.tf`. The SKU (Stock Keeping Unit) represents the pricing tier or plan for your App Service. It defines the performance, features, and capacity of the App Service.
64+
The SKU settings for all Service Plans are defined in the file located at `/infra/variables.tf`. The SKU (Stock Keeping Unit) represents the pricing tier or plan for your App Service. It defines the performance, features, and capacity of the App Service.
7065
More information can be found [here.](https://azure.microsoft.com/en-us/pricing/details/app-service/windows/#purchase-options)
7166

7267
## Web App Service Plan SKU
7368

69+
**File Location:** `/infra/variables.tf`
7470

75-
**File Location:** `/infra/main.tf`
76-
77-
#### SKU Settings
78-
79-
- **Name:** `S1`
80-
- **Capacity:** `3`
71+
### SKU Settings
8172

73+
- **appServiceSkuSize** `S1`
74+
- **appServiceSkuTier** `Standard`
8275

8376
## Functions Service Plan SKU
8477

78+
**File Location:** `/infra/variables.tf`
8579

86-
**File Location:** `/infra/main.tf`
87-
88-
#### SKU Settings
80+
### SKU Settings
8981

90-
- **Name:** `S2`
91-
- **Capacity:** `2`
82+
- **functionsAppSkuSize** `S2`
83+
- **functionsAppSkuTie:** `Standard`
9284

9385
## Enrichment App Service Plan SKU
9486

87+
**File Location:** `/infra/variables.tf`
9588

96-
**File Location:** `/infra/main.tf`
89+
### SKU Settings
9790

98-
#### SKU Settings
99-
100-
- **Name:** `P1v3`
101-
- **Tier:** `PremiumV3`
102-
- **Size:** `P1v3`
103-
- **Family:** `Pv3`
104-
- **Capacity:** `1`
91+
- **enrichmentAppServiceSkuSize** `P1v3`
92+
- **enrichmentAppServiceSkuTier** `PremiumV3`
10593

10694
### Enrichment Message Dequeue Parameter
10795
There exist a property that can be set in the local.env file called `DEQUEUE_MESSAGE_BATCH_SIZE` and is defaulted in the `infra/main.tf` and `app/enrichment/app.py` to the value of **3**. This means the app will process 3 messages from the queue at a time. This is found to be the most optimal with the existing configuration but can be increased if you also increase the enrichment app service SKU. It is important to note that there will be issues if it is increased more than the app service SKU can handle.
@@ -114,30 +102,25 @@ This can also be adjusted in the Azure Portal.
114102

115103
**Note:** Adjusting the scale or Tier can cause outages until the redeployment occurs.
116104

117-
118-
### Steps to Scale Up:
105+
### Steps to Scale Up
119106

120107
>1. **Sign in to the Azure Portal:**
121108
> - Open a web browser and navigate to the [Azure Portal](https://portal.azure.com/).
122109
> - Log in with your Azure account credentials.
123-
124110
>2. **Navigate to the App Service:**
125111
> - In the left navigation pane, select "App Services."
126112
> - Click on the specific App Service you want to scale.
127-
128113
>3. **Access the Scale Up Blade:**
129114
> - In the App Service menu, find and click on "Scale up (App Service plan)" in the left sidebar.
130-
131115
>4. **Choose a New Pricing Tier:**
132116
> - On the "Scale Up" blade, you'll see different pricing tiers representing various levels of resources.
133117
> - Select the desired pricing tier that corresponds to the scale you need.
134-
135118
>5. **Review and Apply Changes:**
136119
> - Review the information about the selected pricing tier, including its features and costs.
137120
> - Click the "Apply" or "Save" button to apply the changes.
138121
122+
### Considerations
139123

140-
### Considerations:
141124
- **Cost Implications:**
142125
- Be aware of the cost implications associated with higher pricing tiers. Review the Azure Pricing documentation for details on costs.
143126

@@ -146,4 +129,3 @@ This can also be adjusted in the Azure Portal.
146129

147130
- **Performance Impact:**
148131
- Scaling up provides additional resources, potentially improving performance. However, it's essential to assess whether your application benefits from the increased resources.
149-

docs/deployment/deployment.md

+6-6
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Once you have the completed setting up a GitHub Codespaces, please move on to th
2929

3030
The IA Accelerator needs to be sized appropriately based on your use case. Please review our [Sizing Estimator](/docs/costestimator.md) to help find the configuration that fits your needs.
3131

32-
To change the size of components deployed, make changes in the [Main Terraform](/infra/main.tf) file.
32+
To change the size of components deployed, make changes in the [Terraform Variables](/infra/variables.tf) file.
3333

3434
Once you have completed the Sizing Estimator and sized your deployment appropriately, please move on to the Configuring your Environment step.
3535

@@ -85,9 +85,9 @@ ENABLE_CUSTOMER_USAGE_ATTRIBUTION <br>CUSTOMER_USAGE_ATTRIBUTION_ID | No | By de
8585
ENABLE_DEV_CODE | No | Defaults to `false`. It is not recommended to enable this flag, it is for development testing scenarios only.
8686
APPLICATION_TITLE | No | Defaults to "". Providing a value for this parameter will replace the Information Assistant's title in the black banner at the top of the UX.
8787
ENTRA_OWNERS | No | Defaults to "". Additional user id's you wish to assign as owners of created Azure Entra objects by way of assign to a security group.
88-
MAX_CSV_FILE_SIZE | No | Defaults to 20 (MB's) for the maximum file size for an uploaded CSV
8988
SERVICE_MANAGEMENT_REFERENCE | No | Defaults to "". Sets the service management reference value on Azure Entra objects created by Information Assistant if required by your organization.
90-
89+
MAX_CSV_FILE_SIZE | Yes | Defaults to 20. This value limits the size of CSV files in MBs that will be supported for upload in the Tabular Data Assistant UX feature.
90+
PASSWORD_LIFETIME | No | Defaults to 365. The number of days that passwords associated with created identities are set to expire after creation. Change this setting if needed to conform to you policy requirements
9191

9292
## Log into Azure using the Azure CLI
9393

@@ -149,13 +149,13 @@ help Show this help
149149
deploy Deploy infrastructure and application code
150150
build Build application code
151151
infrastructure Deploy infrastructure
152-
extract-env Extract infrastructure.env file from TF output
152+
extract-env Extract infrastructure.env file from Terraform output
153153
deploy-webapp Deploys the web app code to Azure App Service
154154
deploy-functions Deploys the function code to Azure Function Host
155155
deploy-enrichments Deploys the web app code to Azure App Service
156156
deploy-search-indexes Deploy search indexes
157-
extract-env-debug-webapp Extract infrastructure.debug.env file from TF output
158-
extract-env-debug-functions Extract local.settings.json to debug functions from TF output
157+
extract-env-debug-webapp Extract infrastructure.debug.env file from Terraform output
158+
extract-env-debug-functions Extract local.settings.json to debug functions from Terraform output
159159
functional-tests Run functional tests to check the processing pipeline is working
160160
merge-databases Upgrade from bicep to terraform
161161
import-state import state of current services to TF state

docs/features/architectural_decisions.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ In Retrieval Augmented Generation applications, a thorough grasp of context is e
2323

2424
Initially, we explored Azure AI Search's built-in skillset for tasks like entity recognition and key phrase extraction. However, due to the additional overhead of utilizing the skillset from Azure AI Search, we opted for custom data processing to extract key phrases and entities such as organizations, locations, and events. This approach enriched the search index by providing additional metadata and context, thereby enhancing retrieval effectiveness. Additionally, we employed embeddings to capture semantic relationships and contextual nuances, improving our understanding of textual data.
2525

26-
To generate embeddings, we empowered users to choose the embedding model that best suits their content and use case, acknowledging that a one-size-fits-all approach is not ideal. Users have the flexibility to opt for the closed-source Azure Open AI embedding or one of the open-source embedding models, including the multilingual embedding model.
26+
To generate embeddings, we empowered users to choose the embedding model that best suits their content and use case, acknowledging that a one-size-fits-all approach is not ideal. Users have the flexibility to opt for the closed-source Azure OpenAI embedding or one of the open-source embedding models, including the multilingual embedding model.
2727

2828
## Document Indexing (Vector Store)
2929

docs/images/app_registration.png

82.1 KB
Loading

docs/images/credential-lifespan.png

79.9 KB
Loading

docs/knownissues.md

+15-1
Original file line numberDiff line numberDiff line change
@@ -206,4 +206,18 @@ Image search is currently only supported with regions that support dense caption
206206
### Solution
207207
These are only in the Logic App Preview Designer. Switching to the Generally Available Designer will resolve these errors. They are purely visual errors in the Preview Designer and have no impact on how the Logic App functions.
208208
209-
![Image of Logic App Error](./images/sharepoint-preview-designer-known-issue.png)
209+
![Image of Logic App Error](./images/sharepoint-preview-designer-known-issue.png)
210+
211+
---
212+
213+
## Error: CredentialInvalidLifetimeAsPerAppPolicy: Credential lifetime exceeds the max value allowed as per assigned
214+
### Solution
215+
Your organization's policy places a limit on the lifetime of an identities password. In your copy of Local.env there is a setting called PASSWORD_LIFETIME. This value is used when creating or updating the identity password and has a default value of the number of days the password will exist before expiring. Change this value to a number of days that your organization allows.
216+
217+
To view the value after deploying go the Microsoft Entra ID page from the Azure Portal home page. Then search your tenant for infoasst_mgmt_access_<your-5-character-suffix> as shown in the image below.
218+
219+
![Image of Entra App Registration](./images/credential-lifespan.png)
220+
221+
Next click on the App Registration value, and then the page will open for that applciuation registration. Then select Clients & Secrets from the left menu. You will then see the expiry date of the password that was applied through Terraform.
222+
223+
![Image of Entra App Registration](./images/app_registration.png)

infra/README.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -24,13 +24,13 @@ help Show this help
2424
deploy Deploy infrastructure and application code
2525
build Build application code
2626
infrastructure Deploy infrastructure
27-
extract-env Extract infrastructure.env file from TF output
27+
extract-env Extract infrastructure.env file from Terraform output
2828
deploy-webapp Deploys the web app code to Azure App Service
2929
deploy-functions Deploys the function code to Azure Function Host
3030
deploy-enrichments Deploys the web app code to Azure App Service
3131
deploy-search-indexes Deploy search indexes
32-
extract-env-debug-webapp Extract infrastructure.debug.env file from TF output
33-
extract-env-debug-functions Extract local.settings.json to debug functions from TF output
32+
extract-env-debug-webapp Extract infrastructure.debug.env file from Terraform output
33+
extract-env-debug-functions Extract local.settings.json to debug functions from Terraform output
3434
functional-tests Run functional tests to check the processing pipeline is working
3535
merge-databases Upgrade from bicep to terraform
3636
import-state import state of current services to TF state

infra/core/aad/entra.tf

+1
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ resource "azuread_application_password" "aad_mgmt_app_password" {
4343
count = var.isInAutomation ? 0 : 1
4444
application_id = azuread_application.aad_mgmt_app[0].id
4545
display_name = "infoasst-mgmt"
46+
end_date_relative = "${var.password_lifetime * 24}h"
4647
}
4748

4849
resource "azuread_service_principal" "aad_mgmt_sp" {

infra/core/aad/variables.tf

+4
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,8 @@ variable "entraOwners" {
4242
variable "serviceManagementReference" {
4343
type = string
4444
sensitive = true
45+
}
46+
47+
variable "password_lifetime" {
48+
type = number
4549
}

infra/main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ module "entraObjects" {
3232
aadMgmtClientSecret = var.aadMgmtClientSecret
3333
entraOwners = var.entraOwners
3434
serviceManagementReference = var.serviceManagementReference
35+
password_lifetime = var.password_lifetime
3536
}
3637

3738
module "logging" {

infra/variables.tf

+6
Original file line numberDiff line numberDiff line change
@@ -512,4 +512,10 @@ variable "entraOwners" {
512512
variable "serviceManagementReference" {
513513
type = string
514514
default = ""
515+
}
516+
517+
variable "password_lifetime" {
518+
type = number
519+
default = 365
520+
description = "The number of days used as the lifetime for passwords"
515521
}

scripts/environments/local.env.example

+4
Original file line numberDiff line numberDiff line change
@@ -151,3 +151,7 @@ export ENTRA_OWNERS=""
151151

152152
# Set a value here if you are required to apply a value in Entra for Service management references
153153
export SERVICE_MANAGEMENT_REFERENCE=""
154+
155+
# A value used in terraform deployment to set the expiry of passwords measure in days.
156+
# Change this setting if needed to conform to you policy requirements
157+
export PASSWORD_LIFETIME=365

scripts/environments/shared-ia-dev.env

+1-3
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,4 @@ export ENABLE_DEV_CODE=false
8888
# Video Indexer API Version used in ARM deployment of Azure Video Indexer
8989
export VIDEO_INDEXER_API_VERSION="2024-01-01"
9090

91-
# If you are using the Bing Search API, you can set the following values to enable safe search.
92-
# Defaults to true if not defined.
93-
export ENABLE_BING_SAFE_SEARCH=true
91+
export PASSWORD_LIFETIME=365

scripts/environments/shared-ia.env

+1-3
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,4 @@ export ENABLE_DEV_CODE=false
8888
# Video Indexer API Version used in ARM deployment of Azure Video Indexer
8989
export VIDEO_INDEXER_API_VERSION="2024-01-01"
9090

91-
# If you are using the Bing Search API, you can set the following values to enable safe search.
92-
# Defaults to true if not defined.
93-
export ENABLE_BING_SAFE_SEARCH=true
91+
export PASSWORD_LIFETIME=365

scripts/environments/tmp-ia.env

+3-1
Original file line numberDiff line numberDiff line change
@@ -86,4 +86,6 @@ export CUSTOMER_USAGE_ATTRIBUTION_ID=""
8686
export ENABLE_DEV_CODE=false
8787

8888
# Video Indexer API Version used in ARM deployment of Azure Video Indexer
89-
export VIDEO_INDEXER_API_VERSION="2024-01-01"
89+
export VIDEO_INDEXER_API_VERSION="2024-01-01"
90+
91+
export PASSWORD_LIFETIME=365

scripts/environments/usgov-ia.env

+1-3
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,4 @@ export ENABLE_DEV_CODE=false
8787
# Video Indexer API Version used in ARM deployment of Azure Video Indexer
8888
export VIDEO_INDEXER_API_VERSION="2024-01-01"
8989

90-
# If you are using the Bing Search API, you can set the following values to enable safe search.
91-
# Defaults to true if not defined.
92-
export ENABLE_BING_SAFE_SEARCH=true
90+
export PASSWORD_LIFETIME=365

scripts/prepare-tf-variables.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -44,4 +44,5 @@ export TF_VAR_enableTabularDataAssistant=$ENABLE_TABULAR_DATA_ASSISTANT
4444
export TF_VAR_enableSharePointConnector=$ENABLE_SHAREPOINT_CONNECTOR
4545
export TF_VAR_enableMultimedia=$ENABLE_MULTIMEDIA
4646
export TF_VAR_maxCsvFileSize=$MAX_CSV_FILE_SIZE
47-
export TF_VAR_serviceManagementReference=$SERVICE_MANAGEMENT_REFERENCE
47+
export TF_VAR_serviceManagementReference=$SERVICE_MANAGEMENT_REFERENCE
48+
export TF_VAR_password_lifetime=$PASSWORD_LIFETIME

0 commit comments

Comments
 (0)