Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource and JNI configuration files to support CRT in GraalVM native image #3122

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

timmattison
Copy link

@timmattison timmattison commented Mar 24, 2022

Motivation and Context

These changes are required to allow customers to build GraalVM native images that utilize the AWS CRT libraries. This may be starting point to address issues raised in #2948.

Modifications

I ran the GraalVM agent to gather the initial JNI configuration. Then I had to tune the results by:

  • Forcing java.nio.Buffer and java.nio.ByteBuffer to include all constructors, methods, and classes
  • Including all resources that end in .so, .dylib, .dll

Testing

I have tested this with Gradle on a separate application that interacts with Greengrass V2's IPC. I believe that it will need some modifications to work in other contexts.

For completeness the only classes that my IPC code references directly in CRT are:

  • software.amazon.awssdk.crt.io.ClientBootstrap
  • software.amazon.awssdk.crt.io.EventLoopGroup
  • software.amazon.awssdk.crt.io.SocketOptions

jdeps provides this list of dependencies it detected:

software.amazon.awssdk.crt.io.ClientBootstrap
software.amazon.awssdk.crt.io.ClientTlsContext
software.amazon.awssdk.crt.io.EventLoopGroup
software.amazon.awssdk.crt.io.HostResolver
software.amazon.awssdk.crt.io.SocketOptions
software.amazon.awssdk.crt.io.SocketOptions$SocketDomain
software.amazon.awssdk.crt.io.SocketOptions$SocketType

So it is possible that applications that depend on CRT classes outside of these may not work.

Some tuning code be done on:

  • java.nio.Buffer and java.nio.ByteBuffer to use only the required elements. I couldn't get this working without everything but there may be some things that could be trimmed.
  • jni-config.json might have extra classes referenced that are not needed. I tried trimming this down but eventually fell back onto the default output of GraalVM's agent.

It is likely Maven configuration will need to be done to pick up the binaries and put them in a place where GraalVM native image can find them. Without building a specific task to copy the binaries to my build/resources directory in Gradle the resource-config.json did not work.

@timmattison timmattison requested a review from a team as a code owner March 24, 2022 14:13
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@timmattison
Copy link
Author

Here is the Gradle snippet I used to bring the binaries into my resources directory to get resource-config.json working

https://gist.github.com/timmattison/f4e1dbb165607c168e8b3f0ca2296d33

@debora-ito
Copy link
Member

Thank you for the submission @timmattison, we'll review it shortly, most likely next week.

Copy link
Contributor

@zoewangg zoewangg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your PR. This is a good start. I wonder if https://github.com/awslabs/aws-crt-java/ is a better place for these files.
For the time being, we can certainly merge this change to a branch and continue to work on it.

@@ -0,0 +1,313 @@
[
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the file to the aws-crt-client module?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have a separate PR for this - awslabs/aws-crt-java#464 - any idea how we can move that one forward?

"methods":[{"name":"onShutdownComplete","parameterTypes":[] }]
},
{
"name":"software.amazon.awssdk.crt.s3.S3MetaRequestProgress",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need S3 related classes in the CRT HTTP client?

@antechrestos
Copy link

Is there any news on it? How can I help, I also met the same problem.. 😢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants