Alias for provider does not work as expected. This `aws.cdn_provider = aws.ssl_provider` does not work.

[EugenKon]

Terraform Version

`Terraform v1.11.3`

Terraform Configuration Files

So I had this configuration at the main level:

provider "aws" {
  alias  = "cdn_provider"
  region = "us-east-1"

  default_tags {
    tags = {
      Project = local.project_name
    }
  }
}

module "private-cloud" {
  source = "./modules/private-cloud"

  ...
  providers = {
    aws.cdn_provider    = aws.cdn_provider
    aws.aws_no_defaults = aws.aws_no_defaults
  }
}

And inside module:

terraform {
  required_providers {
    acme = {
      source  = "vancluever/acme"
      version = "~> 2.26"
    }

    aws = {
      source = "hashicorp/aws"
      configuration_aliases = [
        aws.cdn_provider, aws.aws_no_defaults,
      ]
    }
  }
}

provider "acme" {
  server_url = "https://acme-v02.api.letsencrypt.org/directory"
}

From the submodule the resources were removed and created another submodule at toop level. Provider was renamed: cdn_provider -> ssl_provider.

I am aware of this problem:

│ Error: Provider configuration not present
│
│ To work with module.private-cloud.aws_acm_certificate_validation.acm-ssl-cdn (orphan) its original provider
│ configuration at provider["registry.terraform.io/hashicorp/aws"].cdn_provider is required, but it has been
│ removed. This occurs when a provider configuration is removed while objects created by that provider still exist
│ in the state. Re-add the provider configuration to destroy
│ module.private-cloud.aws_acm_certificate_validation.acm-ssl-cdn (orphan), after which you can remove the provider
│ configuration again.

Thus terraform.required_providers block was not removed from submodule.

Because provider was renamed at top level I changed how it is passed to submodule:

  providers = {
    aws.cdn_provider    = aws.ssl_provider
    aws.aws_no_defaults = aws.aws_no_defaults
  }

But this does not work, submodule still looking for cdn_provider to exists.

The next is very surprising. If I still pass ssl_provider into submodule, but just define cnd_provider at top level without passing it to submodule the submodule "magically" sees cdn_provider though aws.ssl_provider is passed.

provider "aws" {
  alias  = "ssl_provider"
  region = "us-east-1"

  default_tags {
    tags = {
      Project = local.project_name
    }
  }
}

provider "aws" {
   alias  = "cdn_provider"
   region = "us-east-1"

   default_tags {
     tags = {
       Project = local.project_name
     }
   }
 }

And this cdn_provider works even if I change its region to the wrong one.

Debug Output

I can send it privately.

Expected Behavior

Terraform should pass correct provider into submodule
This config should pass ssl_provider into submodule as cdn_provider even if another provider at top level with cdn_provider name still exists.

    aws.cdn_provider    = aws.ssl_provider
    aws.aws_no_defaults = aws.aws_no_defaults

Submodule should not see the names of provider at top level. Submodule should see only the names defined at providers block.

Actual Behavior

TF passes the previous provider into submodule even if the different provider was passed.

Steps to Reproduce

1. create configuration with X provider
2. Pass X provider into submodule
3. create another Y provider
4. pass Y as X into submodule: providers { aws.X = aws.Y }
5. terraform plan/apply. You will get error here
6. define X provider at the top level
7. terraform plan/apply. Even if Y passed to submodule as X, the submodule "magically" sees X provider from the top level.

Additional Context

No response

References

No response

Generative AI / LLM assisted development?

No response

[jbardin]

Provider configurations must always outlive the resources which uses those providers. In order to accommodate the removal of modules within Terraform, the actual providers used for each resource are recorded to ensure the resource can be destroyed even after a module is removed. Passing a provider blindly into a module without a resource to reference that provider doesn't do anything, because the only remaining record of which provider the resource used is a static value in the state.

Generally any provider passed into a module cannot be changed in conjunction with the removal of resources from the module, because the state will always reference the provider which last modified those resources. If you want to move resource between modules, moved blocks not only transfers the state, but can help transfer the configured provider as well.

[EugenKon]

Hi, @jbardin. You miss my point.

Inside module the provider is stored as cdn_provider right?

So I am passing the same provider into module as aws.cdn_provider = XXXX.
The only thing changed on top level is that XXXX was renamed from cdn_provider to ssl_provider.

And this old provider with the new name is passed into module via cdn_provider variable: aws.cdn_provider = aws.ssl_provider.

The module should not distinguish the names of provider in the top level.

1. the actual providers used for each resource are recorded to ensure the resource can be destroyed even after a module is removed.

Some one from HashCorp not so long time ago told me that provider is not stored at the state. And when I remove all resources from the module I can not remove definition of the provider from the module, because terraform plan/apply will not work. Once these resources will be actually destroyed then I can remove required_providers definition from module.

Actually here you state opposite :-/

[jbardin]

Inside module the provider is stored as cdn_provider right?

Modules are not stored in state, but the resource instance recorded that the provider's configuration address was provider["registry.terraform.io/hashicorp/aws"].cdn_provider from the root module, which is stored in the state.

So I am passing the same provider into module as aws.cdn_provider = XXXX.
The only thing changed on top level is that XXXX was renamed from cdn_provider to ssl_provider.

if the resource was removed form the configuration already, then it can't know about this re-assignment, because it no longer exists in the configuration.

The module should not distinguish the names of provider in the top level.

Unfortunately that's not how the system was designed. Because configuration of resources need to be independent of provider configuration, the absolute address of the provider is recorded. Changing that would be a major redesign of how providers are configured, which is being considered for some future new major version, and it was done differently for the new stacks runtime, but for now we need to maintain compatibility with the existing config.

Some one from HashCorp not so long time ago told me that provider is not stored at the state. And when I remove all resources from the module I can not remove definition of the provider from the module, because terraform plan/apply will not work. Once these resources will be actually destroyed then I can remove required_providers definition from module.
Actually here you state opposite :-/

Nope, but the details matter. The provider configuration address is recorded so that we know which provider should be used for destroy operations, but the configuration itself is never stored in the state and must be read from the config files.

[EugenKon]

@jbardin Thanks for explanations. Now it is better but still not clear.

So when users write: https://developer.hashicorp.com/terraform/language/meta-arguments/module-providers#modules-with-alternate-provider-configurations

provider "aws" {
  alias  = "usw1"
  region = "us-west-1"
}

provider "aws" {
  alias  = "usw2"
  region = "us-west-2"
}

module "tunnel" {
  source    = "./tunnel"
  providers = {
    aws.src = aws.usw1
    aws.dst = aws.usw2
  }
}

They should define required_providers as the next (Please add this part into documention):

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      configuration_aliases = [
        aws.src, aws.dst,
      ]
    }
  }
}

1. Here configuration_aliases is very confusing. It does not alias. These are providers the module expects to be defined by main configuration. It would be better to name it as expected_providers.

2. As far as I can see we can not redefine source for providers. Eg. we can not write different sources for the same provider in different modules. In the second module we can not write:

   terraform {
     required_providers {
       aws = {
         source = "different_source/aws"
         configuration_aliases = [
           aws.src, aws.dst,
         ]
       }
     }
   }
   

   If so, it should be documented. Until this moment I thought that for provider["registry.terraform.io/hashicorp/aws"].cdn_provider TF will use info from module, because source is defined there. And thus cdn_provider name (for the example above it is src and dst) will be used. But TF uses source from module and name from main top level.
   If the above is true, then it would be better to define source at provider block at top level.

3. The provider configuration address is recorded so that we know which provider should be used for destroy operations

   It is still very unclear to me (and this is not documented). If TF stores the provider at the state. Why it is required to keep provider definition at the configuration until the first terraform plan/apply when all resources which refers to it were removed from configuration.
   It would be nice to document actions users should perform to delete resources with provider option.

PS. I am not looking for help, this is just feedback about confusing things I experience and how it would be nice to resolve them.

[jbardin]

re: 1. The names are aliases, they are local aliases for the provider being passed in from the parent module. It's still the same provider from the parent, but locally referred to by a name in configuration_aliases.

The source+version attributes in required_providers don't define a new instance of anything, they only represent the external identity of the provider which the module is compatible with. The "instance" of a provider is passed into a module originating from a provider block.

Yes, this can be confusing, but is far better than what existed prior to Terraform v1. Having helped countless users through fixing provider configuration over the years, we've at least reached a happy local-minimum of confusion, where adding more detail tends to derail more users. I think the rest of the documentation you need is included in https://developer.hashicorp.com/terraform/language/modules/develop/providers