Improve error messages for circular x-module references

[VenelinMartinov]

What happened?

In Terraform, two modules are allowed to refer to each other:

modA {
  prop = modB.X
}

modB {
  prop = modA.Y
}

In Pulumi that might be impossible and the error message is not nice:

error: main.pp:95,1-129,2: circular reference; 
error: main.pp:139,63-76: unknown property 'cluster_name' among [id urn]; 
error: main.pp:140,38-52: unknown property 'oidc_provider' among [urn id]; 
error: Detected that /Users/vvm/.pulumi/plugins/resource-terraform-module-v0.1.4/pulumi-resource-terraform-module exited prematurely. 
       This is *always* a bug in the provider. Please report the issue to the provider author as appropriate.
       To assist with debugging we have dumped the STDOUT and STDERR streams of the plugin:

Example

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.47.0"
    }

    random = {
      source = "hashicorp/random"
    }

    tls = {
      source = "hashicorp/tls"
    }

    cloudinit = {
      source = "hashicorp/cloudinit"
    }
  }
}


provider "aws" {
  region = "us-east-1"
}

# Filter out local zones, which are not currently supported 
# with managed node groups
data "aws_availability_zones" "available" {
  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

locals {
  cluster_name = "education-eks-${random_string.suffix.result}"
}

resource "random_string" "suffix" {
  length  = 8
  special = false
}

//@pulumi-terraform-module vpc
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.8.1"

  name = "education-vpc"

  cidr = "10.0.0.0/16"
  azs  = slice(data.aws_availability_zones.available.names, 0, 3)

  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }
}

//@pulumi-terraform-module eks
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "20.8.5"

  cluster_name    = local.cluster_name
  cluster_version = "1.29"

  cluster_endpoint_public_access           = true
  enable_cluster_creator_admin_permissions = true

  cluster_addons = {
    aws-ebs-csi-driver = {
      service_account_role_arn = module.irsa-ebs-csi.iam_role_arn
    }
  }

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_group_defaults = {
    ami_type = "AL2_x86_64"
  }

  eks_managed_node_groups = {
    one = {
      name = "node-group-1"

      instance_types = ["t3.small"]

      min_size     = 1
      max_size     = 3
      desired_size = 2
    }

    two = {
      name = "node-group-2"

      instance_types = ["t3.small"]

      min_size     = 1
      max_size     = 2
      desired_size = 1
    }
  }
}


# https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons/ 
data "aws_iam_policy" "ebs_csi_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

//@pulumi-terraform-module irsa-ebs-csi
module "irsa-ebs-csi" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
  version = "5.39.0"

  create_role                   = true
  role_name                     = "AmazonEKSTFEBSCSIRole-${module.eks.cluster_name}"
  provider_url                  = module.eks.oidc_provider
  role_policy_arns              = [data.aws_iam_policy.ebs_csi_policy.arn]
  oidc_fully_qualified_subjects = ["system:serviceaccount:kube-system:ebs-csi-controller-sa"]
}

Output of pulumi about

.

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Thank you for reporting! Per our discussion Pulumi solution lacks sufficient primitives to solve this problem in the current architecture but we could improve the error message.

In your example, is there a recommended workaround for users? It appears the essential cycle is based on irsa-ebs-csi needing module.eks.oidc_provider and then eks needing module.irsa-ebs-csi.iam_role_arn presumably exposed because of create_role = true. Perhaps the workaround here is to abandon create_role = true and manually provision a role to break the cycle?

[t0yv0]

I've reclassified as enhancement because I believe the bug of supporting x-module references is at "won't fix" status.

[VenelinMartinov]

I think the general workaround is to wrap the two modules in a third TF module and invoke that perhaps? There might also be, as you suggested, case-specific workarounds.

[t0yv0]

That's a nice idea I have not thought through! Yeah this is excellent as it would push down the problem into TF proper.