Azure single server to flexible server

.markdownlint.json

@@ -13,4 +13,4 @@
   "MD033": false,
   "MD036": false,
   "blank_lines": false
-}
+}

CHANGELOG.md

@@ -19,6 +19,7 @@ BUG FIXES:
 * Airlock: Creating an import/export request causes a routing error ([#3830](https://github.com/microsoft/AzureTRE/issues/3830))
 * Fix registration of templates with no 'authorizedRoles' or 'required' defined ([#3849](https://github.com/microsoft/AzureTRE/pull/3849))
 * Update terraform for services bus to move network rules into namespace resource to avoid depreciation warning, and update setup_local_debugging.sh to use network_rule_sets ([#3858](https://github.com/microsoft/AzureTRE/pull/3858))
+* Update terraform MySQL resources to MySQL Flexible resources to fix depricating recources. ([#3892](https://github.com/microsoft/AzureTRE/pull/3892))
 
 COMPONENTS:
 

README.md

@@ -3,6 +3,7 @@
 **Azure TRE documentation site**: <https://microsoft.github.io/AzureTRE/>
 
 ## Background
+
 <img align="right" src="./docs/assets/azure-tre-logo.svg" width="33%" />
 
 Across the health industry, be it a pharmaceutical company interrogating clinical trial results, or a public health provider analyzing electronic health records, there is the need to enable researchers, analysts, and developers to work with sensitive data sets.
@@ -18,14 +19,15 @@ This project is typically implemented alongside a data platform that provides re
 TREs are not “one size fits all”, hence although the Azure TRE has a number of out of the box features, the project has been built be extensible, and hence tooling and data platform agnostic.
 
 Core features include:
+
 - Self-service workspace management for TRE administrators
 - Self-service provisioning of research tooling for research teams
 - Package and repository mirroring - PyPi, R-CRAN, Apt and more.
 - Extensible architecture - build your own service templates as required
 - Azure Active Directory integration
 - Airlock - import and export
 - Cost reporting
-- Ready to workspace templates including:  
+- Ready to workspace templates including:
   - Restricted with data exfiltration control
   - Unrestricted for open data
 - Ready to go workspace service templates including:
@@ -36,7 +38,7 @@ Core features include:
 
 ## Project Status and Support
 
-***This project's code base is still under development and breaking changes will happen. Whilst the maintainers will do our best to minimise disruption to existing deployments, this may not always be possible. Stable releases will be published when the project is more mature.***
+**_This project's code base is still under development and breaking changes will happen. Whilst the maintainers will do our best to minimise disruption to existing deployments, this may not always be possible. Stable releases will be published when the project is more mature._**
 
 The aim is to bring together learnings from past customer engagements where TREs have been built into a single reference solution. This is a solution accelerator aiming to be a great starting point for a customized TRE solution. You're encouraged to download and customize the solution to meet your requirements
 
@@ -46,7 +48,7 @@ It is important before deployment of the solution that the [Support Policy](SUPP
 
 ## Contributing
 
-This project welcomes contributions and suggestions.  Most contributions require you to agree to a
+This project welcomes contributions and suggestions. Most contributions require you to agree to a
 Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
 the rights to use your contribution. For details, visit [https://cla.opensource.microsoft.com](https://cla.opensource.microsoft.com).
 
@@ -66,7 +68,6 @@ This project may contain trademarks or logos for projects, products, or services
 Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
 Any use of third-party trademarks or logos are subject to those third-party's policies.
 
-
 ## Repository structure
 
 ```text

docs/tre-templates/shared-services/gitea.md

@@ -47,3 +47,7 @@ Gitea needs to be able to access the following resource outside the Azure TRE VN
 | AzureActiveDirectory | Authorize the signed in user against Azure Active Directory. |
 | AzureContainerRegistry | Pull the Gitea container image, as it is located in Azure Container Registry.  |
 | (www.)github.com | Allows Gitea to mirror any repo on GitHub |
+
+## Upgrading to version 1.0.0
+
+Migrating existing Gitea services to the major version 1.0.0 is not currently supported. This is due to the breaking change in the Terraform to migrate from the deprecated mysql_server to the new mysql_flexible_server.

docs/tre-templates/workspace-services/gitea.md

@@ -24,3 +24,7 @@ The Gitea worskpace service opens outbound access to:
 3. Click sign in with OpenID button and sign in with the same credentials used to access the workspace.
 4. Once succesfully signed in choose a username.
 5. Navigate to the user settings and under the account tab set a password for your account( `https://<gitea_url>/user/settings/account` ). This username and passowrd should be used to authenticate against Gitea when carrying out git operations.
+
+## Upgrading to version 1.0.0
+
+Migrating existing Gitea services to the major version 1.0.0 is not currently supported. This is due to the breaking change in the Terraform to migrate from the deprecated mysql_server to the new mysql_flexible_server.

docs/tre-templates/workspace-services/mysql.md

@@ -0,0 +1,22 @@
+# MySQL Workspace Service
+
+See: [MySQL Azure](https://learn.microsoft.com/en-GB/azure/mysql/)
+
+## Prerequisites
+
+- [A base workspace deployed](../workspaces/base.md)
+
+- The MySQL workspace service container image needs building and pushing:
+
+  `make workspace_service_bundle BUNDLE=mysql`
+
+## Authenticating to MySQL
+
+1. Navigate to the MySQL workspace service using the `Mysql fqdn` from the details tab.
+2. Using the Password found in Key Vault and the Username `mysqladmin`
+3. Connect to the MySQL server on a VM with the following command shown below
+   `mysql -h [fqdn] -u [username] -p [password]`
+
+## Upgrading to version 1.0.0
+
+Migrating existing MySQL services to the major version 1.0.0 is not currently supported. This is due to the breaking change in the Terraform to migrate from the deprecated mysql_server to the new mysql_flexible_server.

templates/shared_services/gitea/parameters.json

@@ -45,6 +45,12 @@
       "source": {
         "env": "ARM_ENVIRONMENT"
       }
+    },
+    {
+      "name": "sql_sku",
+      "source": {
+        "env": "SQL_SKU"
+      }
     }
   ]
 }

templates/shared_services/gitea/porter.yaml

@@ -1,7 +1,7 @@
 ---
 schemaVersion: 1.0.0
 name: tre-shared-service-gitea
-version: 0.6.10
+version: 1.0.1
 description: "A Gitea shared service"
 dockerfile: Dockerfile.tmpl
 registry: azuretre
@@ -51,6 +51,9 @@ parameters:
     env: ARM_ENVIRONMENT
     type: string
     default: "public"
+  - name: sql_sku
+    type: string
+    default: "B | 4GB 2vCores"
 
 mixins:
   - terraform:
@@ -89,6 +92,7 @@ install:
         mgmt_resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         acr_name: ${ bundle.parameters.mgmt_acr_name }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }
@@ -109,6 +113,7 @@ upgrade:
         mgmt_resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         acr_name: ${ bundle.parameters.mgmt_acr_name }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }
@@ -129,6 +134,7 @@ uninstall:
         mgmt_resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         acr_name: ${ bundle.parameters.mgmt_acr_name }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }

templates/shared_services/gitea/template_schema.json

@@ -27,6 +27,19 @@
       "default": "The Gitea shared service is used for mirroring external Git repositories. For instructions on how to create Git mirrors see [https://docs.gitea.com/usage/repo-mirror](https://docs.gitea.com/usage/repo-mirror).",
       "updateable": true
     },
+    "sql_sku": {
+      "$id": "#/properties/sql_sku",
+      "type": "string",
+      "title": "MySQL server SKU",
+      "description": "MySQL server SKU",
+      "updateable": true,
+      "enum": [
+        "B | 4GB 2vCores",
+        "GP | 8GB 2vCores",
+        "BC | 16GB 2vCores"
+      ],
+      "default": "B | 4GB 2vCores"
+    },
     "is_exposed_externally": {
       "$id": "#/properties/is_exposed_externally",
       "type": "boolean",
@@ -149,4 +162,4 @@
       }
     ]
   }
-}
+}

templates/shared_services/gitea/terraform/gitea-webapp.tf

@@ -46,9 +46,9 @@ resource "azurerm_linux_web_app" "gitea" {
 
     GITEA__database__SSL_MODE = "true"
     GITEA__database__DB_TYPE  = "mysql"
-    GITEA__database__HOST     = azurerm_mysql_server.gitea.fqdn
-    GITEA__database__NAME     = azurerm_mysql_database.gitea.name
-    GITEA__database__USER     = "${azurerm_mysql_server.gitea.administrator_login}@${azurerm_mysql_server.gitea.fqdn}"
+    GITEA__database__HOST     = azurerm_mysql_flexible_server.gitea.fqdn
+    GITEA__database__NAME     = azurerm_mysql_flexible_database.gitea.name
+    GITEA__database__USER     = azurerm_mysql_flexible_server.gitea.administrator_login
     GITEA__database__PASSWD   = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.db_password.id})"
   }
 

templates/shared_services/gitea/terraform/locals.tf

@@ -6,6 +6,11 @@ locals {
   keyvault_name            = "kv-${var.tre_id}"
   version                  = replace(replace(replace(data.local_file.version.content, "__version__ = \"", ""), "\"", ""), "\n", "")
   gitea_allowed_fqdns_list = distinct(compact(split(",", replace(var.gitea_allowed_fqdns, " ", ""))))
+  sql_sku = {
+    "B | 4GB 2vCores"   = { value = "B_Standard_B2s" },
+    "GP | 8GB 2vCores"  = { value = "GP_Standard_D2ds_v4" },
+    "BC | 16GB 2vCores" = { value = "MO_Standard_E2ds_v4" }
+  }
   tre_shared_service_tags = {
     tre_id                = var.tre_id
     tre_shared_service_id = var.tre_resource_id

templates/shared_services/gitea/terraform/mysql.tf

@@ -6,31 +6,25 @@ resource "random_password" "password" {
   min_special = 2
 }
 
-resource "azurerm_mysql_server" "gitea" {
-  name                              = "mysql-${var.tre_id}"
-  resource_group_name               = local.core_resource_group_name
-  location                          = data.azurerm_resource_group.rg.location
-  administrator_login               = "mysqladmin"
-  administrator_login_password      = random_password.password.result
-  sku_name                          = "GP_Gen5_2"
-  storage_mb                        = 5120
-  version                           = "8.0"
-  auto_grow_enabled                 = true
-  backup_retention_days             = 7
-  geo_redundant_backup_enabled      = false
-  infrastructure_encryption_enabled = false
-  public_network_access_enabled     = false
-  ssl_enforcement_enabled           = true
-  ssl_minimal_tls_version_enforced  = "TLS1_2"
-  tags                              = local.tre_shared_service_tags
+resource "azurerm_mysql_flexible_server" "gitea" {
+  name                         = "mysql-${var.tre_id}"
+  resource_group_name          = local.core_resource_group_name
+  location                     = data.azurerm_resource_group.rg.location
+  administrator_login          = "mysqladmin"
+  administrator_password       = random_password.password.result
+  sku_name                     = local.sql_sku[var.sql_sku].value
+  version                      = "8.0.21"
+  backup_retention_days        = 7
+  geo_redundant_backup_enabled = false
+  tags                         = local.tre_shared_service_tags
 
-  lifecycle { ignore_changes = [tags, threat_detection_policy] }
+  lifecycle { ignore_changes = [tags, zone] }
 }
 
-resource "azurerm_mysql_database" "gitea" {
+resource "azurerm_mysql_flexible_database" "gitea" {
   name                = "gitea"
   resource_group_name = local.core_resource_group_name
-  server_name         = azurerm_mysql_server.gitea.name
+  server_name         = azurerm_mysql_flexible_server.gitea.name
   charset             = "utf8"
   collation           = "utf8_unicode_ci"
 }
@@ -41,15 +35,15 @@ moved {
 }
 
 resource "azurerm_private_endpoint" "private_endpoint" {
-  name                = "pe-${azurerm_mysql_server.gitea.name}"
+  name                = "pe-${azurerm_mysql_flexible_server.gitea.name}"
   location            = data.azurerm_resource_group.rg.location
   resource_group_name = local.core_resource_group_name
   subnet_id           = data.azurerm_subnet.shared.id
   tags                = local.tre_shared_service_tags
 
   private_service_connection {
-    private_connection_resource_id = azurerm_mysql_server.gitea.id
-    name                           = "psc-${azurerm_mysql_server.gitea.name}"
+    private_connection_resource_id = azurerm_mysql_flexible_server.gitea.id
+    name                           = "psc-${azurerm_mysql_flexible_server.gitea.name}"
     subresource_names              = ["mysqlServer"]
     is_manual_connection           = false
   }
@@ -63,7 +57,7 @@ resource "azurerm_private_endpoint" "private_endpoint" {
 }
 
 resource "azurerm_key_vault_secret" "db_password" {
-  name         = "${azurerm_mysql_server.gitea.name}-administrator-password"
+  name         = "${azurerm_mysql_flexible_server.gitea.name}-administrator-password"
   value        = random_password.password.result
   key_vault_id = data.azurerm_key_vault.keyvault.id
   tags         = local.tre_shared_service_tags

templates/shared_services/gitea/terraform/variables.tf

@@ -33,3 +33,7 @@ variable "acr_name" {
 variable "arm_environment" {
   type = string
 }
+
+variable "sql_sku" {
+  type = string
+}

templates/workspace_services/gitea/parameters.json

@@ -63,6 +63,12 @@
       "source": {
         "env": "ARM_ENVIRONMENT"
       }
+    },
+    {
+      "name": "sql_sku",
+      "source": {
+        "env": "SQL_SKU"
+      }
     }
   ]
-}
+}

templates/workspace_services/gitea/porter.yaml

@@ -1,7 +1,7 @@
 ---
 schemaVersion: 1.0.0
 name: tre-workspace-service-gitea
-version: 0.8.7
+version: 1.0.1
 description: "A Gitea workspace service"
 dockerfile: Dockerfile.tmpl
 registry: azuretre
@@ -60,6 +60,9 @@ parameters:
     env: ARM_ENVIRONMENT
     type: string
     default: "public"
+  - name: sql_sku
+    type: string
+    default: "B | 4GB 2vCores"
   - name: aad_authority_url
     type: string
     default: "https://login.microsoftonline.com"
@@ -102,6 +105,7 @@ install:
         mgmt_resource_group_name: ${ bundle.parameters.mgmt_resource_group_name }
         aad_authority_url: ${ bundle.parameters.aad_authority_url }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }
@@ -124,6 +128,7 @@ upgrade:
         mgmt_resource_group_name: ${ bundle.parameters.mgmt_resource_group_name }
         aad_authority_url: ${ bundle.parameters.aad_authority_url }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }
@@ -146,6 +151,7 @@ uninstall:
         mgmt_resource_group_name: ${ bundle.parameters.mgmt_resource_group_name }
         aad_authority_url: ${ bundle.parameters.aad_authority_url }
         arm_environment: ${ bundle.parameters.arm_environment }
+        sql_sku: ${ bundle.parameters.sql_sku }
       backendConfig:
         resource_group_name: ${ bundle.parameters.tfstate_resource_group_name }
         storage_account_name: ${ bundle.parameters.tfstate_storage_account_name }

templates/workspace_services/gitea/template_schema.json

@@ -27,6 +27,19 @@
       "description": "Long form description of the workspace service, in markdown syntax",
       "default": "Gitea is a lightweight DevOps platform. Features include code hosting, code review, CI/CD, project management and package management. Documentation can be found here: [https://docs.gitea.com/](https://docs.gitea.com/).",
       "updateable": true
+    },    
+    "sql_sku": {
+      "$id": "#/properties/sql_sku",
+      "type": "string",
+      "title": "MySQL server SKU",
+      "description": "MySQL server SKU",
+      "updateable": true,
+      "enum": [
+        "B | 4GB 2vCores",
+        "GP | 8GB 2vCores",
+        "BC | 16GB 2vCores"
+      ],
+      "default": "B | 4GB 2vCores"
     },
     "is_exposed_externally": {
       "$id": "#/properties/is_exposed_externally",

templates/workspace_services/gitea/terraform/gitea-webapp.tf

@@ -56,9 +56,9 @@ resource "azurerm_linux_web_app" "gitea" {
     GITEA__service__SHOW_REGISTRATION_BUTTON         = false
     GITEA__database__SSL_MODE                        = "true"
     GITEA__database__DB_TYPE                         = "mysql"
-    GITEA__database__HOST                            = azurerm_mysql_server.gitea.fqdn
-    GITEA__database__NAME                            = azurerm_mysql_database.gitea.name
-    GITEA__database__USER                            = "${azurerm_mysql_server.gitea.administrator_login}@${azurerm_mysql_server.gitea.fqdn}"
+    GITEA__database__HOST                            = azurerm_mysql_flexible_server.gitea.fqdn
+    GITEA__database__NAME                            = azurerm_mysql_flexible_database.gitea.name
+    GITEA__database__USER                            = azurerm_mysql_flexible_server.gitea.administrator_login
     GITEA__database__PASSWD                          = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.db_password.id})"
   }
 

templates/workspace_services/gitea/terraform/locals.tf

@@ -8,6 +8,11 @@ locals {
   core_resource_group_name       = "rg-${var.tre_id}"
   keyvault_name                  = lower("kv-${substr(local.workspace_resource_name_suffix, -20, -1)}")
   version                        = replace(replace(replace(data.local_file.version.content, "__version__ = \"", ""), "\"", ""), "\n", "")
+  sql_sku = {
+    "B | 4GB 2vCores"   = { value = "B_Standard_B2s" },
+    "GP | 8GB 2vCores"  = { value = "GP_Standard_D2ds_v4" },
+    "BC | 16GB 2vCores" = { value = "MO_Standard_E2ds_v4" }
+  }
   workspace_service_tags = {
     tre_id                   = var.tre_id
     tre_workspace_id         = var.workspace_id