Core key vault firewall should not be set to "Allow public access from all networks"

[jonnyry]

Resolves #4250

What is being addressed

• Changes the core key vault firewall from Allow public access from all networks to Allow public access from specific virtual networks and IP addresses
• Adds an IP exception to the key vault firewall for the deployment machine's internet IP (or the PUBLIC_DEPLOYMENT_IP_ADDRESS variable if set) during deployment
• Removes the IP exception at the end of deployment (whether deployment succeeds or fails)

How is this addressed

• A new script to add and remove the keyvault deployment IP exception:

  • devops/scripts/kv_add_network_exception.sh

• They are called from the following scenarios in order to provider access to KV:

  • core/terraform/deploy.sh
  • core/terraform/scripts/letsencrypt.sh
  • devops/scripts/destroy_env_no_terraform.sh
  • core/terraform/destroy.sh

• The script uses a bash trap so that it runs regardless of whether the preceeding code fails or not, to ensure the IP exception is removed

A bug in azurerm provider was encountered which required the use of a terraform provisioner:

1. A create provisioner on azurerm_key_vault was required to work around an azurerm provider bug which means if a key vault is being re-created (it was previously soft deleted), the network acls are not updated. This can be removed when the bug is fixed, or a different workaround found.

---

Updates since inital commit (as discussed with @marrobi):

1. Remove use of tags and null provisioner to add tag.
2. Delete the following scripts as they're no longer used:

• devops/scripts/key_vault_list.sh
• devops/scripts/set_contributor_sp_secrets.sh

3. Refactor the add and remove scripts into a single script

[github-actions]

Unit Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit f2c8c96.

♻️ This comment has been updated with latest results.

[jonnyry]

/test 8af920d

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12660338621 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

/test-extended 8af920d

[github-actions]

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/12661150197 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

/test-destroy-env

[github-actions]

Destroying PR test environment (RG: rg-tre26f9d939)... (run: https://github.com/microsoft/AzureTRE/actions/runs/12669260987)

[jonnyry]

/test 2970a5d

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12669597448 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

/test 272589f

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12670289419 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

/test bf9fd32

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12670349633 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

/test-destroy-env

[github-actions]

Destroying PR test environment (RG: rg-tre26f9d939)... (run: https://github.com/microsoft/AzureTRE/actions/runs/12670413797)

[github-actions]

PR test environment destroy complete (RG: rg-tre26f9d939)

[jonnyry]

/test

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/13230152187 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[marrobi]

Tested locally, LGTM

[yuvalyaron]

Nice work!

[jonnyry]

/test-destroy-env

[github-actions]

Destroying PR test environment (RG: rg-tre26f9d939)... (run: https://github.com/microsoft/AzureTRE/actions/runs/13306311442)

[github-actions]

PR test environment destroy complete (RG: rg-tre26f9d939)

[jonnyry]

/test f2c8c96

[github-actions]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/13306353906 (with refid 26f9d939)

(in response to this comment from @jonnyry)

[jonnyry]

🤖 pr-bot 🤖

🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/13306353906 (with refid 26f9d939)

(in response to this comment from @jonnyry)

@marrobi another unexpected "resource already exists error". Performed a /test-destroy-env just prior

Can just force approve this one if you are ok with that?

 »»» 🤖 Creating resource group and storage account...
Location    Name
----------  -------------------
eastus2     ***
WARNING: A storage account with the provided name *** is found. Will continue to update the existing account.
ERROR: (StorageAccountAlreadyTaken) The storage account named *** is already taken.
Code: StorageAccountAlreadyTaken
Message: The storage account named *** is already taken.
make: *** [Makefile:34: bootstrap] Error 1

[marrobi]

@jonnyry I think it takes a bit of time for the storage name to become available again, it can get cached at the Azure platform side for a while. If run again it will likely work.

[marrobi]

/test-force-approve

Tested locally.

[github-actions]

🤖 pr-bot 🤖

✅ Marking tests as complete (for commit f2c8c96)

(in response to this comment from @marrobi)

[jonnyry]

@jonnyry I think it takes a bit of time for the storage name to become available again, it can get cached at the Azure platform side for a while. If run again it will likely work.

ah right, yes DNS issues