Extra data in `Cluster.core` is causing the Java SDK to throw an exception

[justinvp]

I ported https://github.com/pulumi/templates/tree/master/kubernetes-aws-python to Java in pulumi/templates#880:

package myproject;

import com.pulumi.Pulumi;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.enums.AuthenticationMode;

public class App {
    public static void main(String[] args) {
        Pulumi.run(ctx -> {
            // Grab some values from the Pulumi configuration (or use default values)
            var config = ctx.config();
            var minClusterSize = config.getInteger("minClusterSize").orElse(3);
            var maxClusterSize = config.getInteger("maxClusterSize").orElse(6);
            var desiredClusterSize = config.getInteger("desiredClusterSize").orElse(3);
            var eksNodeInstanceType = config.get("eksNodeInstanceType").orElse("t3.medium");
            var vpcNetworkCidr = config.get("vpcNetworkCidr").orElse("10.0.0.0/16");

            // Create a VPC for the EKS cluster
            var eksVpc = new Vpc("eks-vpc", VpcArgs.builder()
                    .enableDnsHostnames(true)
                    .cidrBlock(vpcNetworkCidr)
                    .build());

            // Create the EKS cluster
            var eksCluster = new Cluster("eks-cluster", ClusterArgs.builder()
                    // Put the cluster in the new VPC created earlier
                    .vpcId(eksVpc.vpcId())
                    // Use the "API" authentication mode to support access entries
                    .authenticationMode(AuthenticationMode.Api)
                    // Public subnets will be used for load balancers
                    .publicSubnetIds(eksVpc.publicSubnetIds())
                    // Private subnets will be used for cluster nodes
                    .privateSubnetIds(eksVpc.privateSubnetIds())
                    // Change configuration values to change any of the following settings
                    .instanceType(eksNodeInstanceType)
                    .desiredCapacity(desiredClusterSize)
                    .minSize(minClusterSize)
                    .maxSize(maxClusterSize)
                    // Do not give the worker nodes public IP addresses
                    .nodeAssociatePublicIpAddress(false)
                    // Change these values for a private cluster (VPN access required)
                    .endpointPrivateAccess(false)
                    .endpointPublicAccess(true)
                    .build());

            // Export some values for use elsewhere
            ctx.export("kubeconfig", eksCluster.kubeconfig());
            ctx.export("vpcId", eksVpc.vpcId());
        });
    }
}

And hit the following exception while trying to deploy it:

    error: Running program [PID: 39496](/opt/homebrew/opt/openjdk/libexec/openjdk.jdk/Contents/Home/bin/java -classpath /opt/homebrew/Cellar/maven/3.9.9/libexec/boot/plexus-classworlds-2.8.0.jar -Dclassworlds.conf=/opt/homebrew/Cellar/maven/3.9.9/libexec/bin/m2.conf -Dmaven.home=/opt/homebrew/Cellar/maven/3.9.9/libexec -Dlibrary.jansi.path=/opt/homebrew/Cellar/maven/3.9.9/libexec/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/Users/justin/Documents/2025/02/08/k8saws org.codehaus.plexus.classworlds.launcher.Launcher -Dorg.slf4j.simpleLogger.defaultLogLevel=warn --no-transfer-progress compile exec:java) failed with an unhandled exception:
    java.lang.UnsupportedOperationException: Convert [com.pulumi.eks.Cluster.core]: Error converting 'com.google.common.collect.RegularImmutableMap' to 'TypeShape{type=class com.pulumi.eks.outputs.CoreData, parameters=[]}'. Expected type 'com.pulumi.eks.outputs.CoreData' (annotated with 'com.pulumi.core.annotations.CustomType') to have a setter annotated with @com.pulumi.core.annotations.CustomType$Setter("autoModeNodeRoleName"), got: storageClasses,cluster,eksNodeAccess,vpcCni,encryptionConfig,nodeGroupOptions,nodeSecurityGroupTags,clusterIamRole,awsProvider,kubeconfig,tags,accessEntries,fargateProfile,endpoint,publicSubnetIds,oidcProvider,provider,instanceRoles,vpcId,clusterSecurityGroup,privateSubnetIds,subnetIds
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:119)
        at com.pulumi.serialization.internal.Converter.convertValue(Converter.java:86)
        at com.pulumi.core.internal.OutputCompletionSource.setValue(OutputCompletionSource.java:95)
        at com.pulumi.deployment.internal.DeploymentImpl$ReadOrRegisterResourceInternal.lambda$completeResourceAsync$1(DeploymentImpl.java:1359)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:690)
        at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:527)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1460)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2036)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)
    Caused by: java.lang.IllegalArgumentException: Expected type 'com.pulumi.eks.outputs.CoreData' (annotated with 'com.pulumi.core.annotations.CustomType') to have a setter annotated with @com.pulumi.core.annotations.CustomType$Setter("autoModeNodeRoleName"), got: storageClasses,cluster,eksNodeAccess,vpcCni,encryptionConfig,nodeGroupOptions,nodeSecurityGroupTags,clusterIamRole,awsProvider,kubeconfig,tags,accessEntries,fargateProfile,endpoint,publicSubnetIds,oidcProvider,provider,instanceRoles,vpcId,clusterSecurityGroup,privateSubnetIds,subnetIds
        at com.pulumi.serialization.internal.Converter.lambda$tryConvertObjectInner$5(Converter.java:315)
        at java.base/java.util.HashMap.forEach(HashMap.java:1430)
        at com.pulumi.serialization.internal.Converter.tryConvertObjectInner(Converter.java:313)
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:115)
        ... 9 more
    error: an unhandled error occurred: '/opt/homebrew/bin/mvn -Dorg.slf4j.simpleLogger.defaultLogLevel=warn --no-transfer-progress compile exec:java' exited with non-zero exit code: 32

    [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!

    Feb 08, 2025 10:20:35 AM com.pulumi.deployment.internal.DeploymentImpl$DefaultEngineLogger errorAsync
    SEVERE: Running program [PID: 39496](/opt/homebrew/opt/openjdk/libexec/openjdk.jdk/Contents/Home/bin/java -classpath /opt/homebrew/Cellar/maven/3.9.9/libexec/boot/plexus-classworlds-2.8.0.jar -Dclassworlds.conf=/opt/homebrew/Cellar/maven/3.9.9/libexec/bin/m2.conf -Dmaven.home=/opt/homebrew/Cellar/maven/3.9.9/libexec -Dlibrary.jansi.path=/opt/homebrew/Cellar/maven/3.9.9/libexec/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/Users/justin/Documents/2025/02/08/k8saws org.codehaus.plexus.classworlds.launcher.Launcher -Dorg.slf4j.simpleLogger.defaultLogLevel=warn --no-transfer-progress compile exec:java) failed with an unhandled exception:
    java.lang.UnsupportedOperationException: Convert [com.pulumi.eks.Cluster.core]: Error converting 'com.google.common.collect.RegularImmutableMap' to 'TypeShape{type=class com.pulumi.eks.outputs.CoreData, parameters=[]}'. Expected type 'com.pulumi.eks.outputs.CoreData' (annotated with 'com.pulumi.core.annotations.CustomType') to have a setter annotated with @com.pulumi.core.annotations.CustomType$Setter("autoModeNodeRoleName"), got: storageClasses,cluster,eksNodeAccess,vpcCni,encryptionConfig,nodeGroupOptions,nodeSecurityGroupTags,clusterIamRole,awsProvider,kubeconfig,tags,accessEntries,fargateProfile,endpoint,publicSubnetIds,oidcProvider,provider,instanceRoles,vpcId,clusterSecurityGroup,privateSubnetIds,subnetIds
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:119)
        at com.pulumi.serialization.internal.Converter.convertValue(Converter.java:86)
        at com.pulumi.core.internal.OutputCompletionSource.setValue(OutputCompletionSource.java:95)
        at com.pulumi.deployment.internal.DeploymentImpl$ReadOrRegisterResourceInternal.lambda$completeResourceAsync$1(DeploymentImpl.java:1359)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:690)
        at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:527)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1460)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2036)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)
    Caused by: java.lang.IllegalArgumentException: Expected type 'com.pulumi.eks.outputs.CoreData' (annotated with 'com.pulumi.core.annotations.CustomType') to have a setter annotated with @com.pulumi.core.annotations.CustomType$Setter("autoModeNodeRoleName"), got: storageClasses,cluster,eksNodeAccess,vpcCni,encryptionConfig,nodeGroupOptions,nodeSecurityGroupTags,clusterIamRole,awsProvider,kubeconfig,tags,accessEntries,fargateProfile,endpoint,publicSubnetIds,oidcProvider,provider,instanceRoles,vpcId,clusterSecurityGroup,privateSubnetIds,subnetIds
        at com.pulumi.serialization.internal.Converter.lambda$tryConvertObjectInner$5(Converter.java:315)
        at java.base/java.util.HashMap.forEach(HashMap.java:1430)
        at com.pulumi.serialization.internal.Converter.tryConvertObjectInner(Converter.java:313)
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:115)

The relevant part:

Convert [com.pulumi.eks.Cluster.core]: Error converting 'com.google.common.collect.RegularImmutableMap' to 'TypeShape{type=class com.pulumi.eks.outputs.CoreData, parameters=[]}'. Expected type 'com.pulumi.eks.outputs.CoreData' (annotated with 'com.pulumi.core.annotations.CustomType') to have a setter annotated with @com.pulumi.core.annotations.CustomType$Setter("autoModeNodeRoleName")

The problem appears to be due to some extra data in Cluster.core (autoModeNodeRoleName) that isn't present in the schema. Since it's not in the schema, the Java SDK doesn't know how to handle deserializing it, so it throws.

I tried pinning the template to an earlier version of pulumi-eks before #1519, but ran into the same error but for some other property (apologies, I no longer have the error message).

I have a fix for the Java SDK to be resilient to this extra data in pulumi/pulumi-java#1673. With that fix, the Java SDK will ignore data it can't handle rather than throwing (consistent with other SDKs).

Opening this issue more for awareness, in case there are places in pulumi-eks that accept values from Cluster.core and may be expecting autoModeNodeRoleName (or other data) and then be surprised that SDKs are dropping it. If we wanted to "fix" this in pulumi-eks, I think the fix would be either schematizing the data, or stripping it before saving to state & outputs.

[rquitales]

On further review, it seems like this is possibly a bug with Pulumi EKS and Java. The Cluster resource contains an output property: autoModeNodeRoleName. It seems like we're not able to serialize this properly in the output object, and the fix to ignore data will mask this error.

Here's an example of how we're using this property:

pulumi-eks/tests/testdata/programs/auto-mode-upgrade/step2/index.ts

Line 49 in 2f36894

export const nodeRoleName = cluster.autoModeNodeRoleName;