fix: Configure APIM with VNet integration to access internal services by Copilot · Pull Request #80 · Azure/terraform-azurerm-avm-ptn-aiml-landing-zone

Copilot · 2026-03-14T00:53:10Z

Original prompt

This section details on the original issue you should resolve

<issue_title>API Management configured in External Mode (virtual_network_type=None) cannot access Azure AI Foundry and internal services inside VNet</issue_title>
<issue_description>The deployed Azure API Management (APIM) instance is currently configured without Virtual Network integration. In the Terraform configuration, APIM is deployed with:

virtual_network_type = "None"
virtual_network_subnet_id = null

This configuration places APIM outside the Virtual Network while enabling public network access. However, several backend services, including Azure AI Foundry and other platform services, are deployed inside the landing zone Virtual Network and exposed through private endpoints.

Because APIM is not integrated into the VNet, it cannot access these internal services. Requests routed through APIM to backend services located within the VNet fail due to network isolation.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes API Management configured in External Mode (virtual_network_type=None) cannot access Azure AI Foundry and internal services inside VNet #78

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Change virtual_network_type from "None" to "Internal" (default) and set virtual_network_subnet_id to use APIMSubnet, enabling APIM to access Azure AI Foundry and other backend services deployed inside the VNet via private endpoints. Add configurable virtual_network_type and public_network_access_enabled variables to apim_definition with input validation. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

mbilalamjad · 2026-03-15T21:23:36Z

@copilot can you please review the 5 failing checks and fix the errors in them

… governance updates Private endpoints are not supported when virtual_network_type is Internal or External (they are unnecessary since APIM is already inside the VNet). Only configure private endpoints when virtual_network_type is None. Also applies AVM governance pre-commit changes (grept migration from .github/copilot-instructions.md to .agents/ directory). Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Clarify that when VNet integration is enabled, APIM directly accesses backend services within the VNet (not via private endpoints). Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-15T21:34:42Z

@copilot can you please review the 5 failing checks and fix the errors in them

Investigated all 5 failing checks:

4 example failures (default, default-byo-vnet, standalone, standalone-byo-vnet): The APIM module validation rejects private endpoints when virtual_network_type is "Internal" or "External". Fixed by making private_endpoints conditional — only configured when virtual_network_type == "None" (when APIM is VNet-integrated, it directly accesses services without needing private endpoints).

1 pr-check failure (grept changes): The AVM governance templates had pending updates (migration from .github/copilot-instructions.md to .agents/ directory). Ran ./avm pre-commit to apply these governance changes.

Both fixes are in commits 59d8ff4 and df9e36d.

…from APIM subnet, add NSG outbound rules, add depends_on for NSG rules Three fixes for APIM Internal VNet deployment failures: 1. Remove firewall route table from APIM subnet when APIM uses VNet integration. Forced-tunneling through the firewall blocks APIM's required outbound access to Azure Storage during deployment. 2. Add NSG outbound rule for Azure Storage (port 443) and inbound rule for Azure Load Balancer health probes (port 6390), required per Azure APIM VNet integration documentation. 3. Add depends_on for NSG rules in the APIM module to prevent race condition where APIM deploys before NSG rules are applied. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T05:49:06Z

@copilot can you please review the failing checks and fix the errors in them

Reviewed the 4 failing checks from run 23124116926:

2 APIM deployment failures (default, standalone):

default: APIM ActivationFailed — couldn't connect to Azure Storage from inside the VNet. Fixed by adding NSG outbound rule allowing port 443 to Storage service tag, plus depends_on for NSG rules to prevent race condition.
standalone: APIM context deadline exceeded — the firewall route table was force-tunneling all APIM subnet traffic through the Azure Firewall, blocking outbound to Azure Storage. Fixed by excluding the APIM subnet from the firewall route table when APIM is in Internal/External VNet mode.

Also added NSG inbound rule for Azure Load Balancer health probes (port 6390), required for APIM VNet mode.

2 transient Azure failures (default-byo-vnet, standalone-byo-vnet): Only KeyVault 401 Unauthorized errors (AKV10046) — Azure-side transient auth issues. APIM deployed successfully in both BYO VNet examples.

Commit 1260279.

The KeyVault 401 Unauthorized errors (AKV10046: Unable to resolve the key used for signature validation) during GetCertificateContacts are caused by Azure RBAC not having propagated before certificate operations begin. The modules already had wait_for_rbac_before_key_operations and wait_for_rbac_before_secret_operations set to 60s, but wait_for_rbac_before_certificate_operations was missing. Added the 60s certificate operations wait to both: - main.genai_services.tf (main KeyVault module) - modules/example_hub_vnet/main.tf (hub KeyVault module) Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

mbilalamjad · 2026-03-16T06:16:32Z

@copilot can you please review the failing checks and fix the errors in them

…bac_before_contact_operations The KeyVault AVM module v0.10.2 does not have a wait_for_rbac_before_certificate_operations variable. The correct argument name is wait_for_rbac_before_contact_operations, which matches the GetCertificateContacts API operation that was failing with 401 errors. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T06:19:41Z

@copilot can you please review the failing checks and fix the errors in them

All 5 failures were caused by the same issue: wait_for_rbac_before_certificate_operations is not a valid argument in the KeyVault AVM module v0.10.2. The correct argument name is wait_for_rbac_before_contact_operations (matching the GetCertificateContacts API operation).

Fixed in both main.genai_services.tf and modules/example_hub_vnet/main.tf in commit 7663c83.

mbilalamjad · 2026-03-16T08:16:54Z

@copilot can you please review the failing checks and fix the errors in them

…erations The KeyVault AVM module's wait_for_rbac_before_secret_operations only creates a time_sleep when both role_assignments AND secrets are defined within the module. Since secrets are created by external jumpvm/buildvm modules, the internal time_sleep was never created. Fix by adding explicit time_sleep resources that depend on the KeyVault role assignments, ensuring RBAC propagation completes before the VM modules attempt to set secrets in the KeyVault. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Address code review feedback to ensure time_sleep resets when RBAC changes occur by tracking both keyvault resource_id and principal_id. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T08:26:11Z

@copilot can you please review the failing checks and fix the errors in them

Reviewed the 3 failing jobs from run 23130664582 (commit 7663c83). The pr-check passed ✓.

2 KeyVault 401 failures (standalone, default-byo-vnet): Root cause identified — the KeyVault module's wait_for_rbac_before_secret_operations is conditionally created (count = length(var.role_assignments) != 0 && length(var.secrets) != 0). Since no secrets are defined in the module (they're created by external jumpvm/buildvm modules), the time_sleep was never created and the 60s wait had no effect.

Fixed by adding explicit time_sleep resources outside the KeyVault module that depend on the RBAC role assignments, and updating jumpvm/buildvm to depend on them. Also added the missing depends_on to buildvm which had no dependency on the KeyVault at all.

1 idempotency failure (standalone-byo-vnet): Azure Search Service returns allowed_ips = [] and tags = {} on re-read, causing plan drift. This is a known Azure provider behavior unrelated to our changes.

Commits 7d08616 and fe13e8c.

mbilalamjad · 2026-03-16T15:28:41Z

@copilot can you please review the failing checks and fix the errors in them

…potf fixes Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T15:35:18Z

@copilot can you please review the failing checks and fix the errors in them

Reviewed the 5 failing checks from run 23134450837:

1 code-related failure (pr-check): Three issues fixed:

tflint: Missing time provider in modules/example_hub_vnet/terraform.tf required_providers (the time_sleep resource I added in the previous commit requires it)
Docs out of date: README.md and modules/example_hub_vnet/README.md needed regeneration for the new time_sleep resources
Mapotf formatting: Minor blank line removal in main.genai_services.tf and modules/example_hub_vnet/main.tf

4 transient Azure infrastructure failures:

standalone + standalone-byo-vnet: CosmosDB RequestTimeout — Azure-side timeout during database account creation
default-byo-vnet: AI Foundry capability host creation failure + KeyVault 401 during destroy
default: KeyVault 401 (AKV10046) on jumpvm secret — RBAC propagation timing (the time_sleep should help with retries)

All code-related issues fixed in commit 21f7df5.

mbilalamjad · 2026-03-16T21:29:16Z

@copilot can you please review the failing checks and fix the errors in them. Also please look into why the default examples is taking so long to complete, it seems to get stuck at api management creation

…s conditional on SKU Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T21:39:23Z

@copilot can you please review the failing checks and fix the errors in them. Also please look into why the default examples is taking so long to complete, it seems to get stuck at api...

Reviewed the 4 failing jobs from run 23151921897. The pr-check passed ✓.

1 APIM timeout (default): Root cause identified — the default APIM SKU is Premium_3 in Internal VNet mode, which takes 2.5+ hours to provision (was still creating at 152 minutes when cancelled). Fixed by:

Adding apim_definition overrides in all 4 examples to use Developer_1 SKU, which deploys in ~30-45 minutes in VNet mode
Making zones conditional on SKU (Developer doesn't support availability zones, only Premium does)

3 transient Azure failures:

standalone + standalone-byo-vnet: CosmosDB RequestTimeout — Azure-side timeout during database account creation
default-byo-vnet: DNS Resolver Inbound Endpoint creation failed with subnet provisioning conflict (Conflict: "Subnet provisioning state is not Succeeded")

Commit a7bab6f.

mbilalamjad · 2026-03-16T23:51:50Z

@copilot can you please review the failing checks and fix the errors in them. Please revert api management sku back to premium so that the errors related availability zones are addressed

…ium SKUs Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot · 2026-03-16T23:59:03Z

@copilot can you please review the failing checks and fix the errors in them. Please revert api management sku back to premium so that the errors related availability zones are address...

All 5 failures had the same root cause: the APIM module validation rejects zones = [] for non-Premium SKUs ("Availability Zones are only supported in the Premium tier").

Fixed in commit d632052:

Reverted all 4 examples back to the default Premium SKU (removed Developer_1 overrides)
Changed zones in main.apim.tf from [] to null for non-Premium SKUs — the APIM module validation accepts null but rejects an empty list

Initial plan

e2c5c71

Copilot AI assigned Copilot and mbilalamjad Mar 14, 2026

Copilot started work on behalf of mbilalamjad March 14, 2026 00:53 View session

microsoft-github-policy-service bot added the Needs: Triage 🔍 Maintainers need to triage still label Mar 14, 2026

Copilot AI requested a deployment to test March 14, 2026 00:57 Waiting

Copilot AI temporarily deployed to empty-no-approval March 14, 2026 00:57 Inactive

Copilot AI requested a deployment to test March 14, 2026 00:57 Waiting

Copilot AI changed the title ~~[WIP] Fix API Management access to Azure AI Foundry and internal services~~ fix: Configure APIM with VNet integration to access internal services Mar 14, 2026

Copilot AI requested a review from mbilalamjad March 14, 2026 01:04

Copilot finished work on behalf of mbilalamjad March 14, 2026 01:04

Copilot AI temporarily deployed to empty-no-approval March 15, 2026 21:12 Inactive

Copilot AI temporarily deployed to test March 15, 2026 21:12 Inactive

Copilot AI had a problem deploying to test March 15, 2026 21:12 Failure

Copilot AI temporarily deployed to test March 15, 2026 21:12 Inactive

Copilot AI had a problem deploying to test March 15, 2026 21:14 Failure

Copilot AI temporarily deployed to empty-no-approval March 15, 2026 21:18 Inactive

Copilot started work on behalf of mbilalamjad March 15, 2026 21:23 View session

Copilot AI and others added 2 commits March 15, 2026 21:31

docs: fix variable description for virtual_network_type

df9e36d

Clarify that when VNet integration is enabled, APIM directly accesses backend services within the VNet (not via private endpoints). Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Copilot finished work on behalf of mbilalamjad March 15, 2026 21:35

Copilot finished work on behalf of mbilalamjad March 16, 2026 05:49

Copilot started work on behalf of mbilalamjad March 16, 2026 05:51 View session

Copilot finished work on behalf of mbilalamjad March 16, 2026 05:58

Copilot started work on behalf of mbilalamjad March 16, 2026 05:59 View session

Copilot finished work on behalf of mbilalamjad March 16, 2026 06:00

Copilot started work on behalf of mbilalamjad March 16, 2026 06:01 View session

Copilot finished work on behalf of mbilalamjad March 16, 2026 06:01

Copilot AI had a problem deploying to test March 16, 2026 06:03 Failure

Copilot AI temporarily deployed to empty-no-approval March 16, 2026 06:03 Inactive

Copilot AI and others added 2 commits March 16, 2026 08:24

fix: improve time_sleep trigger in hub_vnet module to track principal_id

fe13e8c

Address code review feedback to ensure time_sleep resets when RBAC changes occur by tracking both keyvault resource_id and principal_id. Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

fix: add time provider to example_hub_vnet, regenerate docs, apply ma…

21f7df5

…potf fixes Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

fix: use Developer_1 APIM SKU in examples for faster CI and make zone…

a7bab6f

…s conditional on SKU Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

fix: revert APIM SKU to Premium and fix zones validation for non-Prem…

d632052

…ium SKUs Co-authored-by: mbilalamjad <79409563+mbilalamjad@users.noreply.github.com>

Conversation

Copilot AI commented Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments on the Issue (you are @copilot in this section)

Uh oh!

mbilalamjad commented Mar 15, 2026

Uh oh!

Copilot AI commented Mar 15, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

mbilalamjad commented Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

mbilalamjad commented Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

mbilalamjad commented Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

mbilalamjad commented Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

mbilalamjad commented Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Mar 14, 2026 •

edited

Loading