azurerm_function_app_flex_consumption re-injects AzureWebJobsStorage into app_settings

[Yakuza-UA]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.11.2

AzureRM Provider Version

4.23.0

Affected Resource(s)/Data Source(s)

azurerm_function_app_flex_consumption

Terraform Configuration Files

resource "azurerm_function_app_flex_consumption" "this" {
  name                = "example-func"
  resource_group_name = var.resource_group.name
  location            = var.resource_group.location
  tags                = var.tags

  service_plan_id = azurerm_service_plan.this.id

  storage_container_endpoint        = join("", [azurerm_storage_account.this.primary_blob_endpoint, azurerm_storage_container.this.name])
  storage_container_type            = "blobContainer"
  storage_authentication_type       = "UserAssignedIdentity"
  storage_user_assigned_identity_id = azurerm_user_assigned_identity.this.id

  runtime_name    = "python"
  runtime_version = "3.10"

  site_config {}

  app_settings = {

    # Environment variables used by functions
    KEY_VAULT_URI                         = azurerm_key_vault.this.vault_uri
    UAMI_CLIENT_ID                        = azurerm_user_assigned_identity.this.client_id
    STORAGE_ENDPOINT                      = azurerm_storage_account.this.primary_blob_endpoint

    # Instruct the function app to use the application insights for logging
    APPLICATIONINSIGHTS_CONNECTION_STRING = azurerm_application_insights.this.connection_string

    # Instruct the function app to use the storage for host state oeprations
    AzureWebJobsStorage__accountname      = azurerm_storage_account.this.name
    AzureWebJobsStorage__clientId         = azurerm_user_assigned_identity.this.client_id
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.this.id]
  }
}

Debug Output/Panic Output

https://gist.github.com/Yakuza-UA/649b7b28353b0f7b20b0bc746378042d

Expected Behaviour

Function App should just work as per Terraform snippet above. It should be configured to use User Assigned Managed Identity to authenticate to the storage account as per:

  storage_container_endpoint        = join("", [azurerm_storage_account.this.primary_blob_endpoint, azurerm_storage_container.this.name])
  storage_container_type            = "blobContainer"
  storage_authentication_type       = "UserAssignedIdentity"
  storage_user_assigned_identity_id = azurerm_user_assigned_identity.this.id

Actual Behaviour

AzureWebJobsStorage is always re-injected into app_settings of the Function App, even though it is configured to use managed identity. The value of the setting is DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=;EndpointSuffix=core.windows.net. This setting forces Func App to use empty access key to authenticate towards storage account, which in turn is RBAC-based authentication and won't ever accept access using access key. I've tried to FORCE it to use managed identity by manually adding

  app_settings = {
    AzureWebJobsStorage__accountname      = azurerm_storage_account.this.name
    AzureWebJobsStorage__clientId         = azurerm_user_assigned_identity.this.client_id
  }

This worked, but only AFTER I've manually deleted AzureWebJobsStorage via the portal, because there's no way to get rid of it via code. Once I deleted this setting my Function App started to work properly. HOWEVER, on the next Terraform plan/apply this setting was re-introduced even though it did not appear in the PLAN output. Once it was re-introduced my Function App stopped working and started logging the following messages:

Host instance '00000000000000000000000066E9C34E' failed to acquire host lock lease: Azure.Storage.Blobs: Service request failed. Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.) ErrorCode: AuthenticationFailed Headers: Transfer-Encoding: chunked Server: Microsoft-HTTPAPI/2.0 x-ms-request-id: 11bd78bf-601e-00e3-3b8b-9a5492000000 x-ms-error-code: AuthenticationFailed Date: Fri, 21 Mar 2025 18:02:37 GMT .

I had to manually delete it again to recover my function app backend.

So, there's misbehaviour of this resource. It should only inject AzureWebJobsStorage when Func App is configured to use Storage account without managed identity and once it is configured to use managed identity (as above) it must NOT inject this setting and instead add the following:

  app_settings = {
    AzureWebJobsStorage__accountname      = azurerm_storage_account.this.name
    AzureWebJobsStorage__clientId         = azurerm_user_assigned_identity.this.client_id
  }

Initially I tried without _clientId but was getting errors as it wasn't able to figure out which managed identity to use, throwing errors about 'being unable to find managed identity'. It may work with system identity which is probably what it tried to attach to, but in my case I use user assigned identity and had to also add _clientId.

Steps to Reproduce

Create function app using azurerm_function_app_flex_consumption and configure it to use USER-ASSIGNED managed to authenticate to the storage account. Once started, check app insights for logs and you will see loads of errors - new one every 10 seconds. I am using Terraform Cloud, hence it is TFC workflow. Also, you will not see any changes to AzureWebJobsStorage in the plan, but it is added at deployment and re-added at each apply.

Important Factoids

Using Terraform Cloud

References

No response

[xiaxyi]

Thanks @Yakuza-UA for raising this issue, the fix is included in the #29099.

The AzureWebJobsStorage should be used when backend storage is using managed identity

[andrewluotechnologies]

I ran into the same issue. For me, I managed to use a temporary workaround (in my case, I am using a managed service identity, but it should also workaround your issue with user-assigned identity):

  app_settings = {
    "AzureWebJobsStorage" = ""
    "AzureWebJobsStorage__accountName" = ...
  }

[tobiasvdp]

The workaround described by Andrew works when using any kind of identity with flex consumption plan.
AzureWebJobsStorage should not be filled in by the provider when using identities.

I ran into the same issue. For me, I managed to use a temporary workaround (in my case, I am using a managed service identity, but it should also workaround your issue with user-assigned identity):

  app_settings = {
    "AzureWebJobsStorage" = ""
    "AzureWebJobsStorage__accountName" = ...
  }

[Yakuza-UA]

@tobiasvdp thanks for this. I've only just got a chance to test it. It works when deployed as you've described.

[ycamargo]

Hi, any updates on fixes for this issue?
I have been looking into the code of other Function App modules, and it looks like the azurerm_linux_function_app module is the one that handles correctly the usage of either AzureWebJobsStorage or AzureWebJobsStorage__accountName .
Also, this module is always injecting the DEPLOYMENT_STORAGE_CONNECTION_STRING variable, event when the connection to the deployment Storage Account is set to use a Managed Identity, it doesn't create an issue but it shouldn't add an unnecessary variable.
Besides overriding the app_settings with an empty AzureWebJobsStorage being a valid workaround for fixing the function startup, it isn't a good solution for those that manages the App Settings outside of Terraform. Because, either we replace all the App Settings every time we run Terraform, or we have to add a lifecycle block to ignore changes to app_settings. The latter works but prevents us from changing the Storage Account in the future, if needed.

[Noel-Jones]

This appears to be the third issue addressing the same underlying problem. I started to look at a fix for #30732 and someone else has started to look at a fix for #29993.