Unable to successfully create VRFs with ndfc_vrfs module

[richwats]

Background:
Nexus Dashboard 4.1.0.138c (dCloud)
Fabric Controller 12.4.1.225

Terraform v1.12.2
on darwin_arm64

• provider registry.terraform.io/ciscodevnet/ndfc v0.1.0

Issue:

I'm trying to create a test VRF using a modified version of example in the documentation and the ndfc_vrfs module

resource "ndfc_vrfs" "test" {
  fabric_name            = "GLOBAL_DC"
  deploy_all_attachments = false
  vrfs = {
    "TEST-VRF1" = {
      vrf_template                   = "Default_VRF_Universal"
      vrf_extension_template         = "Default_VRF_Extension_Universal"
      vrf_id                         = 50030
      vlan_id                        = 3030
      vlan_name                      = "VLAN3030"
      interface_description          = "My int description"
      vrf_description                = "My vrf description"
      mtu                            = 9216
      loopback_routing_tag           = 11111
      redistribute_direct_route_map  = "FABRIC-RMAP-REDIST"
      max_bgp_paths                  = 2
      max_ibgp_paths                 = 3
      ipv6_link_local                = false
      no_rp                          = false
      rp_external                    = true
      overlay_multicast_groups       = "234.0.0.0/8"
      mvpn_inter_as                  = false
      trm_bgw_msite                  = true
      advertise_host_routes          = true
      advertise_default_route        = false
      configure_static_default_route = false
      bgp_password                   = "1234567890ABCDEF"
      bgp_password_type              = "7"
      netflow                        = false
      netflow_monitor                = "MON1"
      disable_rt_auto                = true
      route_target_import            = "1:1"
      route_target_export            = "1:1"
      route_target_import_evpn       = "1:1"
      route_target_export_evpn       = "1:1"
      route_target_import_cloud_evpn = "1:1"
      route_target_export_cloud_evpn = "1:1"
      deploy_attachments             = false
      # attach_list = {
      #   "SWITCH_SERIAL_NO" = {
      #     vlan                   = 1500
      #     deploy_this_attachment = false
      #     loopback_id            = 101
      #     loopback_ipv4          = "1.2.3.4"
      #     loopback_ipv6          = "2001::1"
      #   }
      # }

    }
  }

}

This technically works, i.e. the VRF is created in ND, but the provider reports the following errors and in any further applies, attempts to destroy and replace the created VRF:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # ndfc_vrfs.test is tainted, so must be replaced
-/+ resource "ndfc_vrfs" "test" {
      ~ id                     = "GLOBAL_DC/[TEST-VRF1]" -> (known after apply)
      ~ vrfs                   = {
          ~ "TEST-VRF1" = {
              + advertise_default_route        = false
              + advertise_host_routes          = true
              + bgp_password                   = "1234567890ABCDEF"
              + bgp_password_type              = "7"
              + configure_static_default_route = false
              + mvpn_inter_as                  = false
              + netflow                        = false
              + netflow_monitor                = "MON1"
              + no_rp                          = false
              + overlay_multicast_groups       = "234.0.0.0/8"
              + rp_external                    = true
              + trm                            = false
              + trm_bgw_msite                  = true
              ~ vrf_status                     = "NA" -> (known after apply)
                # (21 unchanged attributes hidden)
            },
        }
        # (2 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

ndfc_vrfs.test: Destroying... [id=GLOBAL_DC/[TEST-VRF1]]
ndfc_vrfs.test: Destruction complete after 0s
ndfc_vrfs.test: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].trm: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].bgp_password_type: was cty.StringVal("7"), but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].no_rp: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].netflow: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].overlay_multicast_groups: was cty.StringVal("234.0.0.0/8"), but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].trm_bgw_msite: was cty.True, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].mvpn_inter_as: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].bgp_password: was cty.StringVal("1234567890ABCDEF"), but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].advertise_default_route: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].netflow_monitor: was cty.StringVal("MON1"), but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].configure_static_default_route: was cty.False, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].advertise_host_routes: was cty.True, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to ndfc_vrfs.test, provider "provider[\"registry.terraform.io/ciscodevnet/ndfc\"]" produced an unexpected new value: .vrfs["TEST-VRF1"].rp_external: was cty.True, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Any help would be greatly appreciated.

[richwats]

[mdmohan]

Hello @richwats ,

Thanks for filing this issue.
The provider is tested only with ND 3.x and not with ND4 and above.

It looks like there is some problem with API backward compatibility on ND4.
Will check this and get back with detailed analysis

Thanks
Murali

[richwats]

Hi @mdmohan , I noticed elsewhere in the provider that Fabric Groups/MSD fabrics aren't supported yet which maybe the cause of the issue as well. I've downgraded my demo lab to ND 3.2 so I may re-try that again too. I'll keep an eye out for ongoing changes as I really need MSD support!

Thank you,
Richard

[mdmohan]

Hi @richwats,

MSD is planned in our next phase and we have started the development.

Is the fabric GLOBAL_DC an MSD parent fabric?
Can you please share the fabric settings also?

Thanks
Murali

[richwats]

Thank you that is good to hear. I think most NDFC customers would have two sites.

Yes it is

Not much needed in the Fabric Group / MSD fabric.

  - BORDER_GWY_CONNECTIONS: Direct_To_BGWS
    DCI_SUBNET_RANGE: 10.100.1.0/24
    FABRIC_NAME: GLOBAL_DC
    FABRIC_TYPE: VXLAN_EVPN_MSD
    LOOPBACK100_IP_RANGE: 10.100.0.0/24
    MS_UNDERLAY_AUTOCONFIG: true
    enableScheduledBackup: true
    scheduledTime: 04:00