[config] Python - generated template is invalid due to lowercase keys in the schema

[menya2]

Python - generated template is invalid due to lowercase keys in the schema.
CfnRemediationConfiguration.RemediationParameterValueProperty, CfnRemediationConfiguration.StaticValueProperty and CfnRemediationConfiguration.ResourceValueProperty results in generating invalid cloudFormation template such that the schema keys start with lower-case letters causing failure to validate and deploy the template. resourceValue, staticValue, value and values (keys) start with lowercase in the generated template output.

Reproduction Steps

s3_encryption_rule = ManagedRule(self, "S3BucketServerSideEncryptionEnabled",
                                         identifier="S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED",
                                         config_rule_name="S3BucketServerSideEncryptionEnabled",
                                         input_parameters={})

        s3_encryption_rule.scope_to_resource("AWS::S3::Bucket")

        automation_assume_role = CfnRemediationConfiguration.RemediationParameterValueProperty(
            static_value=CfnRemediationConfiguration.StaticValueProperty(values=["arn:aws:iam::" + Aws.ACCOUNT_ID + ":role/AutoRemediationRole"]))

        resource_value = CfnRemediationConfiguration.RemediationParameterValueProperty(
            resource_value=CfnRemediationConfiguration.ResourceValueProperty(value="RESOURCE_ID"))

        sse_algorithm = CfnRemediationConfiguration.RemediationParameterValueProperty(
            static_value=CfnRemediationConfiguration.StaticValueProperty(values=["AES256"]))

        remediation = CfnRemediationConfiguration(s3_encryption_rule,
                                                  id="AutoRemediationForS3EncryptionRule",
                                                  config_rule_name="S3BucketServerSideEncryptionEnabled",
                                                  target_id="AWS-EnableS3BucketEncryption",
                                                  target_type="SSM_DOCUMENT",
                                                  target_version="1",
                                                  automatic=True,
                                                  maximum_automatic_attempts=5,
                                                  retry_attempt_seconds=60,
                                                  parameters={"AutomationAssumeRole": automation_assume_role,
                                                              "BucketName": resource_value, "SSEAlgorithm": sse_algorithm})

Error Log

cdk deploy:

Property validation failure: [Encountered unsupported properties in {/Parameters/AutomationAssumeRole}: [staticValue], Encountered unsupported properties in {/Parameters/BucketName}: [resourceValue], Encountered unsupported properties in {/Parameters/SSEAlgorithm}: [staticValue]]

Environment

• CLI Version : 2.0.27
• Framework Version: aws cdk 1.50.0 (build 84acc92)
• Node.js Version: v14.5.0
• OS : Mac OS X Catalina 10.15.5
• Language (Version): Python/3.8.3

Other

The following is getting generated in the CFN template:

"resourceValue": {
              "value": "RESOURCE_ID"
            }

"staticValue": {
              "values": [
                "AES256"
              ]
            }

resourceValue, staticValue, value and values start with lowercase in the generated template output which is causing the error.
Please note that if I change the mentioned keys in the generated template to: ResourceValue, StaticValue, Value and Values
the validation and deployment works as expected hence the cloud-formation template becomes valid.

The following function works as a workaround in Python (converting the values afterwards which is not ideal obviously:

def normalised_dict(d):
            normalised_data = {}
            for k, v in d.items():
                if isinstance(v, dict):
                    v = normalised_dict(v)
                normalised_data[k[0].upper()+k[1:]] = v
            return normalised_data

---

This is 🐛 Bug Report

[menya2]

The issue is still present in version: 1.56.0 (build c1c174d)
I also tested on Windows 10 OS and the same issue was faced.

[MrArnoldPalmer]

Cloudformation documents the type of paramters as JSON which cdk converts to any in typescript. JSON doesn't seem like a correct typing if properties of the object have specific named types. Will have to look into this further to be sure.

[sboardwell]

We also bumped into this with the latest 1.87.1 release. In our case we are adding a managed autoscaler to an EMR cluster.

The following...

            CfnCluster.ComputeLimitsProperty(
                maximum_capacity_units=4,
                minimum_capacity_units=3,
                maximum_core_capacity_units=2,
                maximum_on_demand_capacity_units=2,
                unit_type="Instances"
            ),

...results in...

              {
                "maximumCapacityUnits": 4,
                "minimumCapacityUnits": 3,
                "unitType": "Instances",
                "maximumCoreCapacityUnits": 2,
                "maximumOnDemandCapacityUnits": 2
              },

...which fails with Encountered unsupported property unitType.

Changing to...

              {
                "MaximumCapacityUnits": 4,
                "MinimumCapacityUnits": 3,
                "UnitType": "Instances",
                "MaximumCoreCapacityUnits": 2,
                "MaximumOnDemandCapacityUnits": 2
              },

...works. Our workaround is to perform a replacement with sed post synth.

  sed -i \
    -e 's/unitType/UnitType/g' \
    -e 's/maximumCapacityUnits/MaximumCapacityUnits/g' \
    -e 's/minimumCapacityUnits/MinimumCapacityUnits/g' \
    -e 's/maximumCoreCapacityUnits/MaximumCoreCapacityUnits/g' \
    -e 's/maximumOnDemandCapacityUnits/MaximumOnDemandCapacityUnits/g' \
    cdk.out/*

[fitzoh]

Still present in 2.X (specifically 2.5.0

[Mavus]

I have done some investigation into this bug to discover how you might fix or workaround it. It is not unique to Python based CDK apps. Creating the same constructs in Typescript will also produce an error. Fortunately there is a workaround for the AWS Config example that doesn't require you to perform post processing on your CloudFormation template. This is all in version 2.15.0 (build 151055e).

Python example based on the original issue author's description:

from aws_cdk import Stack
import aws_cdk.aws_config as config

from constructs import Construct

class CdkConfigBugPythonStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        s3_encryption_rule = config.ManagedRule(self, "S3BucketServerSideEncryptionEnabled",
            identifier=config.ManagedRuleIdentifiers.S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED,
        )

        automation_assume_role = config.CfnRemediationConfiguration.RemediationParameterValueProperty(
            static_value=config.CfnRemediationConfiguration.StaticValueProperty(values=["arn:aws:iam:::role/AutoRemediationRole"]))
        resource_value = config.CfnRemediationConfiguration.RemediationParameterValueProperty(
            resource_value= config.CfnRemediationConfiguration.ResourceValueProperty(value="RESOURCE_ID"))
        sse_algorithm = config.CfnRemediationConfiguration.RemediationParameterValueProperty(
            static_value=config.CfnRemediationConfiguration.StaticValueProperty(values=["AES256"]))

        remediation = config.CfnRemediationConfiguration(s3_encryption_rule, id="AutoRemediationForS3EncryptionRule",
            config_rule_name=s3_encryption_rule.config_rule_name,
            target_id="AWS-EnableS3BucketEncryption",
            target_type="SSM_DOCUMENT",
 
            parameters={
                "AutomationAssumeRole": automation_assume_role,
                "BucketName": resource_value,
                "SSEAlgorithm": sse_algorithm
            }
        )

This will produce the following Resources block when running cdk synth:

S3BucketServerSideEncryptionEnabled6C186181:
    Type: AWS::Config::ConfigRule
    Properties:
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
    Metadata:
      aws:cdk:path: CdkConfigBugPythonStack/S3BucketServerSideEncryptionEnabled/Resource
  S3BucketServerSideEncryptionEnabledAutoRemediationForS3EncryptionRule0286604B:
    Type: AWS::Config::RemediationConfiguration
    Properties:
      ConfigRuleName:
        Ref: S3BucketServerSideEncryptionEnabled6C186181
      TargetId: AWS-EnableS3BucketEncryption
      TargetType: SSM_DOCUMENT
      Parameters:
        AutomationAssumeRole:
          staticValue:
            values:
              - arn:aws:iam:::role/AutoRemediationRole
        BucketName:
          resourceValue:
            value: RESOURCE_ID
        SSEAlgorithm:
          staticValue:
            values:
              - AES256

Attempting to deploy this stack with cdk deploy will result in the following error from CloudFormation:
Property validation failure: [Encountered unsupported properties in {/Parameters/BucketName}: [resourceValue], Encountered unsupported properties in {/Parameters/SSEAlgorithm}: [staticValue]]

The same constructs in Typescript will produce the same error

import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as config from 'aws-cdk-lib/aws-config';

export class CdkConfigBugTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const s3EncryptionRule = new config.ManagedRule(this, 'AccessKeysRotated', {
      identifier: config.ManagedRuleIdentifiers.S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED,
    });

    const automationAssumeRoleProperty: config.CfnRemediationConfiguration.RemediationParameterValueProperty = {
      staticValue: {
        values: ['arn:aws:iam:::role/AutoRemediationRole'],
      },
    };
    
    const bucketNameResourceProperty: config.CfnRemediationConfiguration.RemediationParameterValueProperty = {
      resourceValue: {
        value: 'RESOURCE_ID',
      },
    };
    const sseAlgorithmProperty: config.CfnRemediationConfiguration.RemediationParameterValueProperty = {
      staticValue: {
        values: ['AES256'],
      },
    };
    const cfnRemediationConfiguration = new config.CfnRemediationConfiguration(this, 'MyCfnRemediationConfiguration', {
      configRuleName: s3EncryptionRule.configRuleName,
      targetId: 'AWS-EnableS3BucketEncryption',
      targetType: 'SSM_DOCUMENT',
    
      parameters: {
        AutomationAssumeRole: automationAssumeRoleProperty,
        BucketName: bucketNameResourceProperty,
        SSEAlgorithm: sseAlgorithmProperty
      }
    });
  }
}

Which produces the following Resources block in your cdk synth output:

  AccessKeysRotatedF52C0526:
    Type: AWS::Config::ConfigRule
    Properties:
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
    Metadata:
      aws:cdk:path: CdkConfigBugTypescriptStack/AccessKeysRotated/Resource
  MyCfnRemediationConfiguration:
    Type: AWS::Config::RemediationConfiguration
    Properties:
      ConfigRuleName:
        Ref: AccessKeysRotatedF52C0526
      TargetId: AWS-EnableS3BucketEncryption
      TargetType: SSM_DOCUMENT
      Parameters:
        AutomationAssumeRole:
          staticValue:
            values:
              - arn:aws:iam:::role/AutoRemediationRole
        BucketName:
          resourceValue:
            value: RESOURCE_ID
        SSEAlgorithm:
          staticValue:
            values:
              - AES256

As mentioned in a previous comment, the keys staticValue, resourceValue, value and values have not been capitalised. The issue appears to be that the structs (in Typescript, classes in Python)

• aws_config.CfnRemediationConfiguration.RemediationParameterValueProperty
• aws_config.CfnRemediationConfiguration.StaticValueProperty
• aws_config.CfnRemediationConfiguration.ResourceValueProperty
  are is not correctly rendered when synthesising the template with PascalCase keys as the CloudFormation api specifies (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-config-remediationconfiguration-remediationparametervalue.html).

Fortunately I have found a workaround, because the type of the parameters attribute in aws_config.CfnRemediationConfigurationProps is any (Optional[Any] in python). Then instead of providing the type documented "a map of strings to RemediationParameterValue" we can instead provide the values as a literal object (dict in python) rather than using the typed Struct or Class. This would look like this in our above examples:

config.CfnRemediationConfiguration(s3_encryption_rule, id="AutoRemediationForS3EncryptionRule",
  config_rule_name=s3_encryption_rule.config_rule_name,
  target_id="AWS-EnableS3BucketEncryption",
  target_type="SSM_DOCUMENT",

  parameters={
    "AutomationAssumeRole": {"StaticValue": {"Values": ["arn:aws:iam:::role/AutoRemediationRole"]}},
    "BucketName": {"ResourceValue": {"Value": "RESOURCE_ID"}},
    "SSEAlgorithm": {"StaticValue": {"Values": ["AES256"]}}
  }
)

and in Typescript:

const cfnRemediationConfiguration = new config.CfnRemediationConfiguration(this, 'MyCfnRemediationConfiguration', {
  configRuleName: s3EncryptionRule.configRuleName,
  targetId: 'AWS-EnableS3BucketEncryption',
  targetType: 'SSM_DOCUMENT',
  parameters: {
    AutomationAssumeRole: {StaticValue: {Values: ['arn:aws:iam:::role/AutoRemediationRole']}},
    BucketName: {ResourceValue: {Value: 'RESOURCE_ID'}},
    SSEAlgorithm: {StaticValue: {Values: ['AES256']}}
});

Unfortunately this is only possible because the parameters key for aws_config.CfnRemediationConfigurationProps allows any type. It will not work for @sboardwell 's example with CfnCluster.ComputeLimitsProperty which is the explicit type for the construct in which it is used.

When it comes to fixing this bug, because the 'layer 1' Constructs are generated dynamically from the CloudFormation resource specification, I'm not sure how easy it is to fix this in the generated constructs. It might be more sensible to create a 'layer 2' construct for remediation that would apply this workaround when handling parameters.