Cannot create storage account with azure_rm_storageaccount when policy which require default network acl Deny is present.

[erlingre]

SUMMARY

When I try to create a storageaccount in Azure with azure_rm_storageaccount it fails due to a policy which require network_acls default action to be set to 'Deny' in my subscription.
The default_action is set to 'Deny' in the ansible task, but it seems like it is not passed to Azure when deploying the resource.

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_storageaccount

ANSIBLE VERSION

$ ansible --version
ansible [core 2.13.4]
  config file = /home/xxx/yyy/ansible/ansible.cfg
  configured module search path = ['/home/xxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/xxx/ansible39/lib/python3.9/site-packages/ansible
  ansible collection location = /home/xxx/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/xxx/ansible39/bin/ansible
  python version = 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

azure.azcollection            1.13.0 

CONFIGURATION

# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all
DEFAULT_REMOTE_USER(/home/xxx/yyy/ansible/ansible.cfg) = ansible
DEFAULT_STDOUT_CALLBACK(/home/xxx/yyy/ansible/ansible.cfg) = debug
HOST_KEY_CHECKING(/home/xxx/yyy/ansible/ansible.cfg) = False

CONNECTION:
==========

paramiko_ssh:
____________
host_key_checking(/home/xxx/yyy/ansible/ansible.cfg) = False
remote_user(/home/xxx/yyy/ansible/ansible.cfg) = ansible

ssh:
___
host_key_checking(/home/xxx/yyy/ansible/ansible.cfg) = False
remote_user(/home/xxx/yyy/ansible/ansible.cfg) = ansible

OS / ENVIRONMENT

RHEL 8.6
ansible-core 2.13.4
azure.azcollection 1.13.0

STEPS TO REPRODUCE

A policy like the following which requires network_acls default_action to be 'Deny' has to be present to reproduce.
The policy under is obfuscated and just an example.
An ansible-task like under will reproduce the error.

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
 - name: Create storage account for boot diagnostics (console/logging)
     azure_rm_storageaccount:
       network_acls:
         bypass: 'AzureServices,Metrics'
         default_action: 'Deny'
       resource_group: '{{ resource_group }}'
       kind: 'StorageV2'
       access_tier: 'Hot'
       minimum_tls_version: 'TLS1_2'
       name: 'bd_{{ vm_name }}_{{ rnum }}'
       type: 'Standard_LRS'
       https_only: 'True'
       allow_blob_public_access: 'False'

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "policyAssignmentID": {
      "value": "xxx-policy-b33047f8-7a0"
    },
    "policyAssignmentDisplayName": {
      "value": "Deny storage account with network ACL default action"
    },
    "policyDefinitionID": {
      "value": "/providers/Microsoft.Management/managementGroups/#XXXXXX#/providers/Microsoft.Authorization/policyDefinitions/1fb87bd7-f224-4131-322e-b1cc07517fee-v1.0.0",
      "metadata": {
        "version": "1.0.0"
      }
    },
    "scope": {
      "value": "/providers/Microsoft.Management/managementGroups/024d6031-5b2e-4a2e-4231-1050e98b324c",
      "metadata": {
        "name": "hybrid in xxx"
      }
    },
    "nonComplianceMessages": {
      "value": [
        {
          "message": "Unsecure storage account configuration - acl can not have unrestricted access"
        }
      ]
    },
    "parameters": {
      "value": {
        "effect": {
          "value": "Deny"
        }
      }
    }
  }
}

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
I expect the storage account to be created as described in the task.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
The ansible-task fails and no storage account is created.
<!--- Paste verbatim command output between quotes -->
```paste below
ansible-playbook -vvv boot_diagnostics.yml 
[...]
The full traceback is:
  File "/tmp/ansible_azure_rm_storageaccount_payload__m1ljl6a/ansible_azure_rm_storageaccount_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_storageaccount.py", line 837, in create_account
  File "/home/xxx/ansible39/lib/python3.9/site-packages/azure/mgmt/storage/v2021_06_01/operations/_storage_accounts_operations.py", line 207, in begin_create
    raw_result = self._create_initial(
  File "/home/xxx/ansible39/lib/python3.9/site-packages/azure/mgmt/storage/v2021_06_01/operations/_storage_accounts_operations.py", line 155, in _create_initial
    raise HttpResponseError(response=response, error_format=ARMErrorFormat)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "access_tier": "Hot",
            "account_type": "Standard_LRS",
            "ad_user": null,
            "adfs_authority_url": null,
            "allow_blob_public_access": false,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": "auto",
            "blob_cors": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "custom_domain": null,
            "force_delete_nonempty": false,
            "https_only": true,
            "kind": "StorageV2",
            "location": null,
            "log_mode": null,
            "log_path": null,
            "minimum_tls_version": "TLS1_2",
            "name": "bd_1231231_43393874",
            "network_acls": {
                "bypass": "AzureServices,Metrics",
                "default_action": "Deny"
            },
            "password": null,
            "profile": null,
            "resource_group": "xyz",
            "secret": null,
            "state": "present",
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "type": "Standard_LRS"
        }
    }
}

MSG:

Failed to create account: (RequestDisallowedByPolicy) Resource 'bd_1231231_43393874' was disallowed by policy. Reasons: 'Unsecure storage account configuration - acl can not have unrestricted access'. See error details for policy resource IDs.
Code: RequestDisallowedByPolicy
Message: Resource 'bd_1231231_43393874' was disallowed by policy. Reasons: 'Unsecure storage account configuration - acl can not have unrestricted access'. See error details for policy resource IDs.
Target: bd_1231231_43393874
Additional Information:Type: PolicyViolation
Info: {
    "evaluationDetails": {
        "evaluatedExpressions": [
            {
                "result": "True",
                "expressionKind": "Field",
                "expression": "type",
                "path": "type",
                "expressionValue": "Microsoft.Storage/storageAccounts",
                "targetValue": "Microsoft.Storage/storageAccounts",
                "operator": "Equals"
            },
            {
                "result": "True",
                "expressionKind": "Field",
                "expression": "Microsoft.Storage/storageAccounts/networkAcls.defaultAction",
                "path": "properties.networkAcls.defaultAction",
                "targetValue": "Deny",
                "operator": "NotEquals"
            },
            {
                "result": "False",
                "expressionKind": "Field",
                "expression": "id",
                "path": "id",
                "expressionValue": "/subscriptions/12e2fcc1-e21b-43ca-ba25-5964a16085f9/resourceGroups/Oracle/providers/Microsoft.Storage/storageAccounts/bd_1231231_43393874",
                "targetValue": "/storageAccounts/xxxserialco",
                "operator": "Contains"
            }
        ],
        "reason": "Unsecure storage account configuration - acl can not have unrestricted access"
    },
    "policyDefinitionId": "/providers/Microsoft.Management/managementGroups/636a9374-47a2-4119-82e3-faf3189746e2/providers/Microsoft.Authorization/policyDefinitions/0fb81bd7-f384-4131-9d69-b2cc07097fae-v1.0.0",
    "policyDefinitionName": "0ab10bd3-f384-4111-9d62-b1c1c03094fee-v1.0.0",
    "policyDefinitionDisplayName": "Storage accounts should restrict network access - v1.0.0",
    "policyDefinitionEffect": "Deny",
    "policyAssignmentId": "/providers/Microsoft.Management/managementGroups/034d6301-1c0e-4a2e-8045-1090e98b394c/providers/Microsoft.Authorization/policyAssignments/xyz-policy-b33247f1-7b1",
    "policyAssignmentName": "xxx-policy-b3104738-7b0",
    "policyAssignmentDisplayName": "Deny storage account with network ACL default action",
    "policyAssignmentScope": "/providers/Microsoft.Management/managementGroups/133d6011-5b0e-5a2e-1045-1090e98b394c",
    "policyAssignmentParameters": {
        "effect": "Deny"
    }
}

[Fred-sun]

@erlingre This error is not a violation of the naming rules caused by the error, the naming rules are as follows, I suggest you change the name to try! Thank you very much!

Storage account name must be between 3 and 24 characters in length and use numbers and lower-case letters only. 

[erlingre]

@Fred-sun Correct observed, but it fails the same way with a correct storage account name:

Failed to create account: (RequestDisallowedByPolicy) Resource 'bdb00dbzl09971620910' was disallowed by policy. Reasons: 'Unsecure storage account configuration - acl can not have unrestricted access'. See error details for policy resource IDs.
Code: RequestDisallowedByPolicy

The ansible task creating the storage account looks like this:

  azure_rm_storageaccount:
    resource_group: '{{ resource_group }}'
    name: 'bd{{ vm_name }}{{ rnum }}'
    type: 'Standard_LRS'
    https_only: 'true'
    allow_blob_public_access: 'false'
    network_acls:
      bypass: AzureServices
      default_action: Deny
      ip_rules:
        value: "10.23.6.0/26"
        action: "Allow"
      virtual_network_rules:
        id: "{{ vnet_name }}"
        action: "Allow"

Also using managed storage for boot diagnostics fails in the same way:

Failed to create storage account: b00dbzl0991596 - (RequestDisallowedByPolicy) Resource 'b00dbzl0991596' was disallowed by policy. Reasons: 'Unsecure storage account configuration - storage blob can not have anonymous(public) access','Unsecure storage account configuration - acl can not have unrestricted access'. See error details for policy resource IDs.
Code: RequestDisallowedByPolicy

  azure.azcollection.azure_rm_virtualmachine:
    admin_username: "{{ admin_username }}"
    managed_disk_type: '{{ managed_disk_type }}'
    name: '{{ vm_name }}'
    network_interface_names: "{{ vm_name }}-nic"
    os_type: Linux
    resource_group: '{{ resource_group }}'
    short_hostname: '{{ vm_name }}'
    vm_size: '{{ vm_size }}'
    vm_identity: SystemAssigned # Enable System Assigned Managed Identity for AAD ssh-logins
    boot_diagnostics: #  Provides Serial Console Access 
      enabled: yes
      resource_group: '{{ resource_group }}'
    ssh_password_enabled: false
    ssh_public_keys:
      - path: /home/{{ admin_username }}/.ssh/authorized_keys
        key_data: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
    image:
      offer: "RHEL"
      publisher: "RedHat"
      sku: "8-LVM"
      version: "latest"

The same ansible code can be used to crate a virtual machine with boot diagnostics in a subscription not enforcing such a policy for the storage account.
Maybe the policy is the problem rather than ansible. Using manage storage I assume that the azure.azcollection.azure_rm_virtualmachine module is not beeing able to do anything regarding network acl's.

[erlingre]

Now I understand better. I assumed that the following ansible code enabled boot diagnostics with managed storage account, but it is creating a custom storage account where the default network acl is set to allow which then causes the policy violation error.

 boot_diagnostics: #  Provides Serial Console Access 
      enabled: yes
      resource_group: '{{ resource_group }}'

Is it possible to change azure_rm_virtualmachine so that enabling boot diagnostics as shown here uses managed storage instead of a custom storage account? Using managed storage for boot diagnostics is also recommended by Microsoft.
If a custom storage account is desired the user can then specify a custom one with the storage_account-option.

[Fred-sun]

@erlingre Have you resolved your above problem of not being able to create a storage account? Should be caused by your sub's policy, can not directly create a storage account! Thanks!

[Fred-sun]

@erlingre As for your second question "Using managed storage for boot diagnostic" and #993 are duplicate, we are considering solving this problem, thank you!

[dhawal1304]

@Fred-sun is this issue fixed, if not is there any workaround.

[Fred-sun]

@dhawal1304 @erlingre Can you run on the last version? I have tested and get successfully result. Thank you!