Dynamic provider configuration assignment

[WilliamABradley]

Current Terraform Version

Terraform v0.13.0-beta1

Use-cases

It would be nice to be able to create dynamic providers. The main reason for my usage would be for aws assume_role. I have Terraform to create a number of AWS Subaccounts, and then I want to configure those subaccounts in one apply, instead of breaking them up across multiple apply steps.

Currently this is done via modules, but with 0.12, I had to manually define copies of the modules for each subaccount.

As said by the 0.13 modules doc: https://github.com/hashicorp/terraform/blob/master/website/docs/configuration/modules.html.md#limitations-when-using-module-expansion

Modules using count or for_each cannot include configured provider blocks within the module. Only proxy configuration blocks are allowed.

Attempted Solutions

# Organisational Group for Crimson App.
module "org_crimson_app" {
  source          = "./modules/organisation-group"
  organisation_id = local.root_organisation_id
  group_name      = "crimson-app"

  billing_alert_emails = ["<billing account>"]

  accounts = {
    production = {}
    staging = {}
    test    = {}
  }
  zones = {
    public = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.org"
    }
    internal = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.io"
    }
  }

  master_role_name       = local.organisation_root_role
  account_owner_username = var.account_owner_username
  account_owner_domain   = var.account_owner_domain

  // Workaround for https://github.com/hashicorp/terraform/issues/17519
  account_configuration = {
    production = module.org_crimson_app_production_config.config
    staging    = module.org_crimson_app_staging_config.config
    test       = module.org_crimson_app_test_config.config
  }
}

// Workaround for https://github.com/hashicorp/terraform/issues/17519
module "org_crimson_app_production_config" {
  source       = "./modules/organisation-account-config"
  account_info = module.org_crimson_app.account_information.production
}

// Workaround for https://github.com/hashicorp/terraform/issues/17519
module "org_crimson_app_staging_config" {
  source       = "./modules/organisation-account-config"
  account_info = module.org_crimson_app.account_information.staging
}

// Workaround for https://github.com/hashicorp/terraform/issues/17519
module "org_crimson_app_test_config" {
  source       = "./modules/organisation-account-config"
  account_info = module.org_crimson_app.account_information.test
}

Source for ./modules/organisation-group:

# The Organisational Unit for this group.
resource "aws_organizations_organizational_unit" "unit" {
  name      = var.group_name
  parent_id = var.organisation_id

  lifecycle {
    prevent_destroy = true
  }
}

# The environment accounts.
resource "aws_organizations_account" "environments" {
  for_each  = var.accounts
  parent_id = aws_organizations_organizational_unit.unit.id

  # Use account_name if provided, otherwise [group]-[account].
  name  = local.account_full_name[each.key]
  email = "${var.account_owner_username}+aws-org-${local.account_full_name[each.key]}@${var.account_owner_domain}"

  role_name = var.master_role_name

  lifecycle {
    # There is no AWS Organizations API for reading role_name
    ignore_changes  = [role_name]
    prevent_destroy = true
  }
}

# Collect all Account Names
locals {
  account_full_name = {
    for account_name, account in var.accounts :
    account_name =>
    lookup(account, "account_name", "${var.group_name}-${account_name}")
  }
}
...

Source for ./modules/organisation-account-config:

// Workaround for https://github.com/hashicorp/terraform/issues/17519
variable "account_info" {
  type = object({
    group_name        = string
    account_name      = string
    account_full_name = string
    alias             = string
    master_role_arn   = string
    zones = map(object({
      zone_name = string
      fqdn      = string
    }))
    add_terraform_bucket   = bool
    billing_alert_amount   = number
    billing_alert_currency = string
    billing_alert_emails   = list(string)
  })
}

# Create Provisioner Assuming Child Account Role.
provider "aws" {
  region = "ap-southeast-2"

  assume_role {
    role_arn = var.account_info.master_role_arn
  }
}

resource "aws_iam_account_alias" "alias" {
  account_alias = var.account_info.alias
}

# Bucket for Terraform State.
resource "aws_s3_bucket" "terraform_state" {
  count = var.account_info.add_terraform_bucket ? 1 : 0

  bucket = "${var.account_info.account_full_name}-terraform"
  versioning {
    enabled = true
  }
}

# Fix for Issue: https://medium.com/faun/aws-eks-the-role-is-not-authorized-to-perform-ec2-describeaccountattributes-error-1c6474781b84
resource "aws_iam_service_linked_role" "elasticloadbalancing" {
  aws_service_name = "elasticloadbalancing.amazonaws.com"
}

...

Proposal

Module for_each:

# Organisational Group for Crimson App.
module "org_crimson_app" {
  source          = "./modules/organisation-group"
  organisation_id = local.root_organisation_id
  group_name      = "crimson-app"

  billing_alert_emails = ["<billing account>"]

  accounts = {
    production = {}
    staging = {}
    test    = {}
  }
  zones = {
    public = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.org"
    }
    internal = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.io"
    }
  }

  master_role_name       = local.organisation_root_role
  account_owner_username = var.account_owner_username
  account_owner_domain   = var.account_owner_domain

  account_configuration = module.org_crimson_app_config
}

module "org_crimson_app_config" {
  for_each = module.org_crimson_app.account_information
  source       = "./modules/organisation-account-config"
  account_info = each.value
}

Provider for_each:

This doesn't look as clean, but appeases the docs saying:

In all cases it is recommended to keep explicit provider configurations only in the root module and pass them (whether implicitly or explicitly) down to descendent modules. This avoids the provider configurations from being "lost" when descendent modules are removed from the configuration. It also allows the user of a configuration to determine which providers require credentials by inspecting only the root module.

# Organisational Group for Crimson App.
module "org_crimson_app" {
  source          = "./modules/organisation-group"
  organisation_id = local.root_organisation_id
  group_name      = "crimson-app"

  billing_alert_emails = ["<billing account>"]

  accounts = {
    production = {}
    staging = {}
    test    = {}
  }
  zones = {
    public = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.org"
    }
    internal = {
      create  = true
      zone_id = ""
      domain  = "app.crimsoneducation.io"
    }
  }

  master_role_name       = local.organisation_root_role
  account_owner_username = var.account_owner_username
  account_owner_domain   = var.account_owner_domain

  account_configuration = module.org_crimson_app_config
}

provider "aws" {
  for_each = module.org_crimson_app.account_information
  alias = each.key

  assume_role {
    role_arn = each.value.master_role_arn
  }
}

module "org_crimson_app_config" {
  for_each = module.org_crimson_app.account_information
  source       = "./modules/organisation-account-config"
  providers = {
    aws = aws[each.key]
  }
  account_info = each.value
}

References

• Another user references a use case: https://discuss.hashicorp.com/t/terraform-v0-13-0-beta-program/9066/9
• Dynamically create providers with for each. #25320

[tenderitaf]

Hi,
is this still under the radar ?
is there any plan to fix this yet ?
thanks

[bflad]

Just to link the issues, see also #19932 for multiple provider instantiation.

[tenderitaf]

can someone from Hashicorp bother himself and answer, Issue is open for a year now!!!
Would be good to fix to answer those RFC in terraform before announcing new product capabilities

[extemporalgenome]

Indeed. Without a workaround feature, this is a fundamental weakness of making post-auth information like region conventionally part of the provider rather than a more normal input to the resource, and as such, terraform becomes unreasonably verbose for expressing non-trivial architectures.

[aaomidi]

This comment may help people here as well: #29840 (comment)

Again, not tested for more than one, but I think it should just work.

[nfx]

Another use-case many practitioners have is for_each on modules with providers. The most recent use-case:

1. base/databricks-common is initializing databricks provider via host & token combinations, that are outputs from either base/aws or base/azure.
2. aws-classroom includes base/aws & base/databricks-common
3. azure-classroom includes base/azure & base/databricks-common

└── modules
    ├── aws-classroom
    │   └── main.tf
    ├── azure-classroom
    │   └── main.tf
    └── base
        ├── aws
        │   ├── main.tf
        │   └── provider.tf
        ├── aws-shared
        │   ├── main.tf
        │   └── provider.tf
        ├── azure
        │   └── main.tf
        ├── databricks-common
        │   ├── main.tf
        │   ├── policy.tf
        │   ├── provider.tf
        │   └── users.tf
        └── defaults
            └── main.tf

Let's simplify the desired outcome of this feature:

data "http" "classrooms" {
  url = "https://internal-ws-based-on-non-tech-users-input/classrooms.json"
}

module "classrooms" {
  for_each      = data.http.classrooms.value
  source         = "./modules/classroom"
  name           = each.value.name
}

resource "aws_s3_bucket_object" "all_classrooms" {
  bucket = "..."
  key    = "classrooms.json"
  content_base64 = base64(json_encode([for c in module.classrooms: {
    "name" = c.name,
    "url" = c.url
  }]))
}

so that we just leave the auto-approve to terraform and run it as background job in Terraform Cloud or anywhere else every 30 minutes, having no bothering to change the HCL files through GIT every time we need to add a new classroom. But this is not possible because of this bug.

some folks have solved the similar problem with code-generating the classroom equivalent modules via bash script, but this cannot really be done as a single-state apply... theoretically we can hack this around via github_repository_file, and use data.http.classrooms.value to generate & change classrooms.tf file that would have nothing but hardcoded module declarations, but it's still a hack.