Deployment Failure When Creating AI Services Account with Private Networking

[scdouble]

• Description:

I'm encountering a deployment failure when deploying a Bicep template. I would greatly appreciate any guidance on how to resolve this issue.
the bicep template is below:
15-private-network-standard-agent-setup

• My Setup:

I manually created the following resources in advance:
VNet: 192.168.0.0/16
Agent Subnet: 192.168.1.0/24
Private Endpoint (PE) Subnet: 192.168.2.0/24
Azure AI Search: Created with PublicNetworkAccess = Disabled
Cosmos DB: Created with PublicNetworkAccess = Disabled
Storage Account: Created with PublicNetworkAccess = Disabled
Region: japaneast
I then provided these resource references in the azuredeploy.json parameters UI.

• Issue:

The deployment fails at the step of creating the AI Services Account (Microsoft.CognitiveServices/accounts). The error message is:

{
  "code": "DeploymentFailed",
  "target": "/subscriptions/733c0ac8-aaa8-4a85-b5d6-b788a74d82de/resourceGroups/rg-local-aif/providers/Microsoft.Resources/deployments/aiservicesy4z2-y4z2-deployment",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
  "details": [
    {
      "code": "ResourceDeploymentFailure",
      "target": "/subscriptions/733c0ac8-aaa8-4a85-b5d6-b788a74d82de/resourceGroups/rg-local-aif/providers/Microsoft.CognitiveServices/accounts/aiservicesy4z2",
      "message": "The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "OperationError",
          "message": "Failed to Create the resource. Provisioning state: Failed",
          "details": [
            {
              "code": "ResourceProviderError",
              "message": "Failed to create Aml RP virtual workspace due to System.Exception: Failed async operation {\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"InternalServerError\"\n  }\n}\n   at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AsyncOperationHelper.PollUntilCompleteAsync[T](TraceContext traceContext, HttpResponseMessage response, HttpClient httpClient, Int32 timeoutInMin, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AsyncOperationHelper.cs:line 117\n   at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AmlRpClient.PutAmlRpVirtualHubAsync(TraceContext traceContext, Dictionary`2 headers, ResourceIdentityInArm resourceIdentity, String primaryUserAssignedIdentity, String internalId, KeyVaultProperties keyVaultProperties, String subscriptionId, String resourceGroupName, String workspaceName, List`1 networkInjections, PublicNetworkAccessType publicNetworkAccessType, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AmlRpClient.cs:line 119\n   at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AmlRpService.PutAmlRpWorkspaceAsync(TraceContext traceContext, ResourceRequestContext requestContext, WorkerMessageRequestContext workerMessageRequestContext, IResourceEntity insertedResourceEntity, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AmlRpService.cs:line 124\n   at Microsoft.CognitiveServices.ResourceProvider.Worker.MessageProcessor.ProcessInternalAsync(TraceContext traceContext, RpWorkerQueueMessage message, CancellationToken cancellationToken) in /__w/1/s/src/ResourceProvider/Rp.WorkerRole/AccountProvisioning/MessageProcessor.cs:line 223"
            }
          ]
        }
      ]
    }
  ]
}

• What I’ve Tried:

Confirmed that all dependent resources (VNet, subnets, private resources) are successfully created.

• Request:

Could you please help identify:
Why this internal server error is occurring during the creation of the AI Services account?
Are there any known issues with private endpoint/Azure ML RP workspace creation in japaneast?
Any workarounds or required configurations I might have missed?

[DarylGraves]

I had the same error in #309 - It turns out my error was because Microsoft.App was not registered against the subscription. I mistakenly assumed it was because we have deployed a number of App Service Plans in the same subscription but I guess this is for something else.

In my situation, I was able to delete the Resource Group, register the providers listed in the Prerequisites here and re-run the deployment using a different name for the resources. Hope this helps.

[jcbartle]

Confirming that @DarylGraves suggested solution does fix this error (at least in certain situations). I didn't have to delete the RG or anything.