DENA-437: kafka-shared: dev: refactor iam to use the tls-app module

[sbuliarca]

Quotas need to be replaced, so there will be some disruption.

The plan is:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.iam_cerbos_audit_exporter_consumer.kafka_acl.group_acl has moved to module.iam_cerbos_audit_exporter.kafka_acl.group_acl["auth.iam-cerbos-audit-v1"]
    resource "kafka_acl" "group_acl" {
        id                           = "User:CN=auth/iam-cerbos-audit-exporter|*|Read|Allow|Group|exporter-iam-cerbos-audit-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_cerbos_audit_exporter_consumer.kafka_acl.topic_acl has moved to module.iam_cerbos_audit_exporter.kafka_acl.topic_acl["auth.iam-cerbos-audit-v1"]
    resource "kafka_acl" "topic_acl" {
        id                           = "User:CN=auth/iam-cerbos-audit-exporter|*|Read|Allow|Topic|auth.iam-cerbos-audit-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_cerbos_audit_exporter.kafka_quota.quota must be replaced
  # (moved from module.iam_cerbos_audit_exporter_consumer.kafka_quota.consumer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "producer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth/iam-cerbos-audit-exporter|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_cerbos_audit_indexer_consumer.kafka_acl.group_acl has moved to module.iam_cerbos_audit_indexer.kafka_acl.group_acl["auth.iam-cerbos-audit-v1"]
    resource "kafka_acl" "group_acl" {
        id                           = "User:CN=auth/iam-cerbos-audit-indexer|*|Read|Allow|Group|indexer-iam-cerbos-audit-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_cerbos_audit_indexer_consumer.kafka_acl.topic_acl has moved to module.iam_cerbos_audit_indexer.kafka_acl.topic_acl["auth.iam-cerbos-audit-v1"]
    resource "kafka_acl" "topic_acl" {
        id                           = "User:CN=auth/iam-cerbos-audit-indexer|*|Read|Allow|Topic|auth.iam-cerbos-audit-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_cerbos_audit_indexer.kafka_quota.quota must be replaced
  # (moved from module.iam_cerbos_audit_indexer_consumer.kafka_quota.consumer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "producer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth/iam-cerbos-audit-indexer|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_credentials_producer_auth_provider.kafka_acl.producer_acl has moved to module.iam_clubhouse_auth_provider.kafka_acl.producer_acl["auth-customer.iam-credentials-v1"]
    resource "kafka_acl" "producer_acl" {
        id                           = "User:CN=clubhouse/auth-provider|*|Write|Allow|Topic|auth-customer.iam-credentials-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_clubhouse_auth_provider.kafka_quota.quota must be replaced
  # (moved from module.iam_credentials_producer_auth_provider.kafka_quota.producer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "consumer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=clubhouse/auth-provider|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_credentials_producer_api.kafka_acl.producer_acl has moved to module.iam_credentials_api.kafka_acl.producer_acl["auth-customer.iam-credentials-v1"]
    resource "kafka_acl" "producer_acl" {
        id                           = "User:CN=auth-customer/credentials-api|*|Write|Allow|Topic|auth-customer.iam-credentials-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_credentials_api.kafka_quota.quota must be replaced
  # (moved from module.iam_credentials_producer_api.kafka_quota.producer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "consumer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth-customer/credentials-api|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_credentials_consumer.kafka_acl.group_acl has moved to module.iam_credentials_indexer.kafka_acl.group_acl["auth-customer.iam-credentials-v1"]
    resource "kafka_acl" "group_acl" {
        id                           = "User:CN=auth-customer/iam-credentials-v1-indexer|*|Read|Allow|Group|indexer-iam-credentials-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_credentials_consumer.kafka_acl.topic_acl has moved to module.iam_credentials_indexer.kafka_acl.topic_acl["auth-customer.iam-credentials-v1"]
    resource "kafka_acl" "topic_acl" {
        id                           = "User:CN=auth-customer/iam-credentials-v1-indexer|*|Read|Allow|Topic|auth-customer.iam-credentials-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_credentials_indexer.kafka_quota.quota must be replaced
  # (moved from module.iam_credentials_consumer.kafka_quota.consumer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "producer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth-customer/iam-credentials-v1-indexer|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_identitydb_identity_api_consumer.kafka_acl.group_acl has moved to module.iam_identity_api.kafka_acl.group_acl["auth.iam-identitydb-v1"]
    resource "kafka_acl" "group_acl" {
        id                           = "User:CN=auth/iam-identity-api|*|Read|Allow|Group|iam-identity-api|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_identitydb_identity_api_consumer.kafka_acl.topic_acl has moved to module.iam_identity_api.kafka_acl.topic_acl["auth.iam-identitydb-v1"]
    resource "kafka_acl" "topic_acl" {
        id                           = "User:CN=auth/iam-identity-api|*|Read|Allow|Topic|auth.iam-identitydb-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_identity_api.kafka_quota.quota must be replaced
  # (moved from module.iam_identitydb_identity_api_consumer.kafka_quota.consumer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "producer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth/iam-identity-api|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_identitydb_event_forwarder_producer.kafka_acl.producer_acl has moved to module.iam_identitydb_event_forwarder.kafka_acl.producer_acl["auth.iam-identitydb-v1"]
    resource "kafka_acl" "producer_acl" {
        id                           = "User:CN=auth/iam-identitydb-event-forwarder|*|Write|Allow|Topic|auth.iam-identitydb-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_identitydb_event_forwarder.kafka_quota.quota must be replaced
  # (moved from module.iam_identitydb_event_forwarder_producer.kafka_quota.producer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "consumer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth/iam-identitydb-event-forwarder|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.iam_identitydb_snapshotter_consumer.kafka_acl.group_acl has moved to module.iam_identitydb_snapshotter.kafka_acl.group_acl["auth.iam-identitydb-v1"]
    resource "kafka_acl" "group_acl" {
        id                           = "User:CN=auth/iam-identitydb-snapshotter|*|Read|Allow|Group|iam-identitydb-snapshotter|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_identitydb_snapshotter_consumer.kafka_acl.topic_acl has moved to module.iam_identitydb_snapshotter.kafka_acl.topic_acl["auth.iam-identitydb-v1"]
    resource "kafka_acl" "topic_acl" {
        id                           = "User:CN=auth/iam-identitydb-snapshotter|*|Read|Allow|Topic|auth.iam-identitydb-v1|Literal"
        # (7 unchanged attributes hidden)
    }

  # module.iam_identitydb_snapshotter.kafka_quota.quota must be replaced
  # (moved from module.iam_identitydb_snapshotter_consumer.kafka_quota.consumer_quota)
-/+ resource "kafka_quota" "quota" {
      ~ config      = { # forces replacement
          + "producer_byte_rate" = 5242880
            # (2 unchanged elements hidden)
        }
      ~ id          = "User:CN=auth/iam-identitydb-snapshotter|user" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

Plan: 8 to add, 0 to change, 8 to destroy.

[linear]

DENA-437 Terraform intermittently failling on shared kafka-cluster

[matthewhughes-uw]

~ id = "User:CN=auth/iam-cerbos-audit-exporter|user" -> (known after apply)

Are these IDs just going to get the same value as before (but Kafka just can't know that yet)? If not, does anything downstream depend on these?

[matthewhughes-uw]

🤔 thought for future: maybe we could add pre-commit to this repo to run terraform fmt for us to keep things tidy

[sbuliarca]

Yea, that would be good

[matthewhughes-uw]

🤔 thought for future: maybe we could add pre-commit to this repo to run terraform fmt for us to keep things tidy

#137

[sbuliarca]

~ id = "User:CN=auth/iam-cerbos-audit-exporter|user" -> (known after apply)

Are these IDs just going to get the same value as before (but Kafka just can't know that yet)? If not, does anything downstream depend on these?

Yes, it will be the same id