Pulumi Preview/Up fails with EKS provider in Deployment Pipelines

[minato7]

Describe what happened

Hi,

We are experiencing issue during pulumi preview/up with usage of EKS provider in Azure Deployment Pipeline. This is not reproducible in local setup but only in the pipeline.

Any help to debug this issue would be appreciated. We tried different ways to preview but just fails with ToString error. Our usage is quite straightforward with a EKS cluster creation.

Sample program

Dotnet program with EKS provider and EKS cluster creation in Azure Deployment Pipeline. Tested this with multiple programs in different pipelines, same error message in all of them.

Log output

pulumi up --diff --non-interactive --yes --refresh=False 
Previewing update (sample):
  pulumi:pulumi:Stack: (sample)
error: eks:index:Cluster resource 'diagnostics-eks-cluster' has a problem: Calling [toString] on an [Output<T>] is not supported.

To get the value of an Output<T> as an Output<string> consider either:
1: o.apply(v => `prefix${v}suffix`)
2: pulumi.interpolate `prefix${v}suffix`

See https://www.pulumi.com/docs/concepts/inputs-outputs for more details.

Or use ESLint with https://github.com/pulumi/eslint-plugin-pulumi to warn or
error lint on using Output<T> in template literals.
error: Error: Calling [toString] on an [Output<T>] is not supported.

To get the value of an Output<T> as an Output<string> consider either:
1: o.apply(v => `prefix${v}suffix`)
2: pulumi.interpolate `prefix${v}suffix`

See https://www.pulumi.com/docs/concepts/inputs-outputs for more details.

Or use ESLint with https://github.com/pulumi/eslint-plugin-pulumi to warn or
error lint on using Output<T> in template literals.
    at Proxy.OutputImpl.toString (/snapshot/eks/node_modules/@pulumi/pulumi/output.js:85:23)
    at Array.join (<anonymous>)
    at Array.toString (<anonymous>)
    at Object.transferProperties (/snapshot/eks/node_modules/@pulumi/pulumi/runtime/rpc.js:95:89)
    at /snapshot/eks/node_modules/@pulumi/pulumi/runtime/resource.js:576:33
    at Generator.next (<anonymous>)
    at /snapshot/eks/node_modules/@pulumi/pulumi/runtime/resource.js:21:71
    at new Promise (<anonymous>)
    at __awaiter (/snapshot/eks/node_modules/@pulumi/pulumi/runtime/resource.js:17:12)
    at prepareResource (/snapshot/eks/node_modules/@pulumi/pulumi/runtime/resource.js:506:12)
error: Error: 14 UNAVAILABLE: No connection established. Last error: connect ECONNREFUSED 127.0.0.1:33029 (2025-06-17T10:56:22.085Z)
    at callErrorFromStatus (/snapshot/eks/node_modules/@grpc/grpc-js/src/call.ts:82:17)
    at Object.onReceiveStatus (/snapshot/eks/node_modules/@grpc/grpc-js/src/client.ts:360:55)
    at Object.onReceiveStatus (/snapshot/eks/node_modules/@grpc/grpc-js/src/client-interceptors.ts:458:34)
    at Object.onReceiveStatus (/snapshot/eks/node_modules/@grpc/grpc-js/src/client-interceptors.ts:419:48)
    at /snapshot/eks/node_modules/@grpc/grpc-js/src/resolving-call.ts:163:24

Affected Resource(s)

No response

Output of pulumi about

CLI
Version 3.177.0
Go Version go1.24.3
Go Compiler gc

Pulumi.Eks Version="3.9.1"

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[blampe]

@minato7 the error message includes a couple tips to help debug this. Did you read the docs it links to?

This error message happens when you use an output as a string. You probably want Apply or Interpolate but without seeing your code I can't say much more than that.

[minato7]

Thank you @blampe Indeed there is a clear instruction which says ToString shall not be used for the cluster name but the code does not perform any of it. Here is the code which creates the cluster. As you can see, there is no output which is used as a string.

   private Cluster CreateEksCluster(string name, EksClusterArgs args)
    {
        var nodeRole = new Role(
            $"{name}-eks-node-role",
            new()
            {
                Name = $"{name}-eks-node-role",
                AssumeRolePolicy = GetAssumeRolePolicy(["sts:AssumeRole"], ["ec2.amazonaws.com"])
            },
            new CustomResourceOptions
            {
                Parent = this,
                IgnoreChanges = IamRole.DefaultIgnoreChanges
            });

        var clusterRole = new Role(
            $"{name}-eks-cluster-role",
            new()
            {
                Name = $"{name}-eks-cluster-role",
                AssumeRolePolicy = GetAssumeRolePolicy(["sts:AssumeRole", "sts:TagSession"], ["eks.amazonaws.com"])
            },
            new CustomResourceOptions
            {
                Parent = this,
                IgnoreChanges = IamRole.DefaultIgnoreChanges
            });

        var policyAttachments = GetClusterClusterPolicy(name, clusterRole, nodeRole);

        var eksCluster = new Cluster(
            $"{name}-eks-cluster",
            new ClusterArgs
            {
                AuthenticationMode = AuthenticationMode.Api,
                CreateOidcProvider = true,
                CorednsAddonOptions = new()
                {
                    Enabled = false
                },
                EndpointPrivateAccess = true,
                EndpointPublicAccess = true,
                InstanceRole = nodeRole,
                InstanceType = InstanceType,
                Name = args.ClusterName,
                ServiceRole = clusterRole,
                SubnetIds = args.PrivateSubnetIds,
                Version = "1.32",
                SkipDefaultNodeGroup = true,
                VpcId = args.VpcId,
                Tags = args.Tags,
            },
            new ComponentResourceOptions
            {
                Parent = this,
                DependsOn = policyAttachments
            });
         
           // some more ops here
           return eksCluster;
     }

[algompluecker]

@blampe please also note that the code of the user program is C#. The error message would be different if the C# code would trigger the Output<>.ToString.
From the pulumi-dotnet:
"1. o.Apply(v => $"prefix{v}suffix")",
"2. Output.Format($"prefix{hostname}suffix");",

[blampe]

@algompluecker that's a very good point!

@minato7 for good measure can you show me how InstanceType is defined and what GetClusterClusterPolicy is doing?

[blampe]

I suspect you're seeing this because we're failing to properly render this error message:

None of the configured subnets are in the provided availability zone ('${zone}'). Choose a different availability zone or change the subnets.

I recommend checking your AZ and subnets, and will separately prepare a fix for the error message.

[minato7]

I suspect you're seeing this because we're failing to properly render this error message:

None of the configured subnets are in the provided availability zone ('${zone}'). Choose a different availability zone or change the subnets.

I recommend checking your AZ and subnets, and will separately prepare a fix for the error message.

Good catch but the original error arises from Cluster creation, not from ManagedNodeGroup which you have pointed out. There is no error during ManagedNodeGroup or it does not even reach that point because it fails at Cluster creation. Here is the sample snippet:

        var nodeRole = new Role(
            $"{name}-eks-node-role",
            new()
            {
                Name = $"{name}-eks-node-role",
                AssumeRolePolicy = GetAssumeRolePolicy(["sts:AssumeRole"], ["ec2.amazonaws.com"])
            },
            new CustomResourceOptions
            {
                Parent = this,
                IgnoreChanges = IamRole.DefaultIgnoreChanges
            });

        var clusterRole = new Role(
            $"{name}-eks-cluster-role",
            new()
            {
                Name = $"{name}-eks-cluster-role",
                AssumeRolePolicy = GetAssumeRolePolicy(["sts:AssumeRole", "sts:TagSession"], ["eks.amazonaws.com"])
            },
            new CustomResourceOptions
            {
                Parent = this,
                IgnoreChanges = IamRole.DefaultIgnoreChanges
            });

        var policyAttachments = GetClusterClusterPolicy(name, clusterRole, nodeRole);

        var eksCluster = new Cluster(
            $"{name}-eks-cluster",
            new ClusterArgs
            {
                AuthenticationMode = AuthenticationMode.Api,
                CreateOidcProvider = true,
                CorednsAddonOptions = new()
                {
                    Enabled = false
                },
                EndpointPrivateAccess = true,
                EndpointPublicAccess = true,
                InstanceRole = nodeRole,
                InstanceType = InstanceType,
                Name = args.ClusterName,
                NodeRootVolumeEncrypted = true,
                ServiceRole = clusterRole,
                SubnetIds = args.PrivateSubnetIds,
                Version = "1.32",
                SkipDefaultNodeGroup = true,
                VpcId = args.VpcId,
                Tags = args.Tags,
            },
            new ComponentResourceOptions
            {
                Parent = this,
                DependsOn = policyAttachments,
            });

        var instanceType = "c5.4xlarge";

        var eksManagedGroup = new ManagedNodeGroup(
            $"{name}-mg-eks-node-group",
            new ManagedNodeGroupArgs
            {
                Cluster = eksCluster,
                EnableIMDSv2 = true,
                NodeRole = nodeRole,
                SubnetIds = args.PrivateSubnetIds,
                InstanceTypes = instanceType,
                ScalingConfig = new NodeGroupScalingConfigArgs()
                {
                    DesiredSize = 1,
                    MaxSize = 2,
                    MinSize = 1
                }
            },
            new ComponentResourceOptions
            {
                Parent = this,
            });

As I said, there is no error during local deployment aka local machine, it fails only in Azure Pipelines deployment, we have the same subnets configured and there is no other difference.