Parameter store as Cloud Config not working in native image on AWS Lambda

[kieranjen]

I am trying to use the parameter store feature explained in the documentation here. When building the application as a native image and trying to run on AWS Lambda, I get an error on startup.

Task List

• Steps to reproduce provided
• Stacktrace (if present) provided
• Example that reproduces the problem uploaded to Github
• Full description of the issue provided (see below)

Steps to Reproduce

1. Build the example application with ./gradlew buildNativeLambda
2. Create an SSM parameter called /config/application/example/awsTest with any value e.g. Test
3. Upload the built native image zip to an AWS Lambda custom runtime
4. Run the lambda function using the following JSON as the lambda input:

{
  "body": "eyJ0ZXN0IjoiYm9keSJ9",
  "resource": "/",
  "path": "/",
  "httpMethod": "POST",
  "isBase64Encoded": true,
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "/"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.eu-west-2.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.eu-west-2.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/",
    "resourcePath": "/",
    "httpMethod": "POST",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

Expected Behaviour

When the application starts it should pick up the configuration from the SSM parameter store and print and return it from the API call.

Actual Behaviour

The following exception is raised when the application is starting up:

io.micronaut.context.exceptions.ConfigurationException: Exception thrown instantiating MicronautLambdaRuntimeHandler
--
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:40)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.createHandler(AbstractMicronautLambdaRuntime.java:192)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.startRuntimeApiEventLoop(AbstractMicronautLambdaRuntime.java:313)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.run(AbstractMicronautLambdaRuntime.java:129)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.main(MicronautLambdaRuntime.java:50)
Caused by: com.amazonaws.serverless.exceptions.ContainerInitializationException: Error starting Micronaut container: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:252)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:162)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:115)
at io.micronaut.function.aws.proxy.MicronautLambdaHandler.<init>(MicronautLambdaHandler.java:54)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:38)
... 4 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1565)
at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:219)
at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:2857)
at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:231)
at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:165)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:248)
... 8 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1972)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2724)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2710)
at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2249)
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1559)
... 13 more
Caused by: java.lang.ExceptionInInitializerError
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:315)
at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:143)
at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:137)
at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:100)
at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:80)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
at io.micronaut.aws.sdk.v1.AWSClientConfiguration.<init>(AWSClientConfiguration.java:36)
at io.micronaut.aws.sdk.v1.$AWSClientConfigurationDefinition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1943)
... 17 more
Caused by: java.lang.IllegalArgumentException
at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:249)
at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:263)
at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:336)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
... 26 more
Request loop failed with: Exception thrown instantiating MicronautLambdaRuntimeHandler

Environment Information

• Operating System: AWS Lambda (native image)
• Micronaut Version: 2.4.2
• JDK Version: java11

Example Application

https://github.com/kieranjen/micronaut-graal-ssm-params

[hchang-chwy]

Seems to be related to AWSParameterStoreConfiguration where it extends AWSClientConfiguration from aws sdk v1 lib which is by nature not compatible with native image.

[graemerocher]

There is a PR waiting on @sdelamo #1060

[hchang-chwy]

Per https://arnoldgalovics.com/tackling-java-cold-startup-times-on-aws-lambda-with-graalvm/, I was finally able to fix the issue by adding below files under src/main/graal

resource-config.json

{
  "resources": [
    {
      "pattern": "com/amazonaws/internal/config/awssdk_config_default.json"
    },
    {
      "pattern": "com/amazonaws/partitions/endpoints.json"
    },
    {
      "pattern": "com/amazonaws/sdk/versionInfo.properties"
    }
  ]
}

reflect.json

[
  {
    "name": "com.amazonaws.internal.config.InternalConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HostRegexToRegionMappingJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.JsonIndex",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  }
]

[graemerocher]

@ilopmar maybe we should include the above config until the PR @sdelamo has assigned to him is dealt with?

[ilopmar]

We do have support for ParameterStore but using SDK v2 (which includes it out of the box). This is our test application: https://github.com/micronaut-graal-tests/micronaut-aws-sdk2-graal/tree/2.5.x_paramstore

[graemerocher]

So what is #1060 for?

[ilopmar]

@sdelamo can confirm it but I think that PR adds support for use Parameter Store as a Distributed Config provider in Micronaut.

[sdelamo]

#1060 is share logic for both distributed configuration solutions AWS Parameter store and AWS Secret Manager.

[graemerocher]

@sdelamo can this be closed since in Micronaut 3 it now uses the v2 SDK?