Graal Native Image Support

[aherlihy]

Hi, does the DuckDB JDBC driver support use from a graal native image?

I tried with v1.1.1 using graalvm-jdk-17.0.12+8.1 using SBT NativeImagePlugin, but it fails with:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: <...>/build/debug/libduckdb_java.so_osx_universal
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryAbsolute(NativeLibrarySupport.java:100)
	at java.base@17.0.12/java.lang.ClassLoader.loadLibrary(ClassLoader.java:57)
	at java.base@17.0.12/java.lang.Runtime.load0(Runtime.java:755)
	at java.base@17.0.12/java.lang.System.load(System.java:1957)
	at org.duckdb.DuckDBNative.<clinit>(DuckDBNative.java:52)
	at org.duckdb.DuckDBConnection.newConnection(DuckDBConnection.java:51)
	at org.duckdb.DuckDBDriver.connect(DuckDBDriver.java:41)
	at java.sql@17.0.12/java.sql.DriverManager.getConnection(DriverManager.java:681)
	at java.sql@17.0.12/java.sql.DriverManager.getConnection(DriverManager.java:252)
        ...

on Ubuntu 22.04 LTS (and equivalently build/debug/libduckdb_java.so_osx_universal on MacOS 15.3.1). It works fine from a regular JAR. The build.sbt native image settings are:

 .settings(
    nativeImageInstalled := true,
    nativeImageGraalHome := {
      val executable = "native-image"
      val paths = System.getenv("PATH").split(Pattern.quote(pathSeparator))
      paths.map(Paths.get(_)).find(p => Files.isExecutable(p.resolve(executable))) match {
        case Some(binDir) =>
          binDir.getParent
        case None =>
          throw new MessageOnlyException(s"Could not find binary `$executable` in PATH.")
      }
    },
    nativeImageCommand := nativeImageCommand.value ++ Seq(
      "--no-fallback",
      "--report-unsupported-elements-at-runtime"
    ),

    libraryDependencies ++= Seq(
       ...,
      "org.duckdb" % "duckdb_jdbc" % "1.1.1",
    )

I tried extracting the library files from the DuckDB jar and linking them manually but it just segfaulted, so I would be very appreciative of any help. Thank you!

[staticlibs]

Hi! AOT is unlikely to work in the current version.

Short term, the idea is to make native library loading more flexible - support loading it from FS without unpacking from JAR, on a default or specified path. When this is working, the Java part, that is pretty thin and does not use reflection, should be easy to compile with Graal. And library loading should be flexible enough to work with Graal.

Long term, the idea (only an idea at this point) to have DuckDB engine (without Java-specific bits) to be packaged as a shared lib that exposes stable C-API. And to have JDBC impl as Java-only (again, without reflection) that uses Foreign Function & Memory API to work with the engine shared lib over C-API. This way the Java part (that will have more logic than now) can be AOT-compiled as an ordinary Java application, assuming Graal will have the full support for FFM.

PS: I will comment here when library loading enhancements are done.