[JDBC] DuckDB JDBC driver SIGSEGV the JVM since 0.9.0

[loicmathieu]

What happens?

Since version 0.9.0, using the DuckDB JDBC driver in a Java application makes the application crash with a SIGSEGV.
The Java version is 17.0.5 (tested also on 17.0.8.1).

There is first a Java exception

java.sql.SQLException: random_device could not be read
	at org.duckdb.DuckDBNative.duckdb_jdbc_startup(Native Method)
	at org.duckdb.DuckDBConnection.newConnection(DuckDBConnection.java:48)
	at org.duckdb.DuckDBDriver.connect(DuckDBDriver.java:38)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:681)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
	at io.kestra.plugin.jdbc.JdbcConnectionInterface.connection(JdbcConnectionInterface.java:63)
	at io.kestra.plugin.jdbc.AbstractJdbcQuery.run(AbstractJdbcQuery.java:77)
	at io.kestra.plugin.jdbc.duckdb.Query.run(Query.java:148)
	at io.kestra.plugin.jdbc.duckdb.Query.run(Query.java:31)
	at io.kestra.core.runners.Worker$WorkerThread.run(Worker.java:674) 	

Then a JVM crash

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f523603fd60, pid=37746, tid=39346
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.5+8 (17.0.5+8) (build 17.0.5+8)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (17.0.5+8, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C  0x00007f523603fd60
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to <redacted>)
#
# An error report file with more information is saved as:
# <redacted>
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

It works well with 0.8.0.

To Reproduce

Here is the SQL query:

      INSTALL httpfs;
      SELECT Title, max("Days In Top 10") 
      from (SELECT * FROM read_parquet('s3://duckdb-md-dataset-121/netflix_daily_top_10.parquet'))
      where Type='Movie'
      GROUP BY Title
      ORDER BY max("Days In Top 10") desc
      limit 5;

The code uses the standard Java JDBC API (Connection & Statement), but it is not easily extracted as it runs via Kestra DuckDB plugin.

OS:

Ubuntu 23.04

DuckDB Version:

0.9.0

DuckDB Client:

Java JDBC

Full Name:

Loïc Mathieu

Affiliation:

Kestra

Have you tried this on the latest main branch?

I have tested with a release build (and could not test with a main build)

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• Yes, I have

[loicmathieu]

See the corresponding core dump log:
hs_err_pid37746.log

[carlopi]

Could you try with v0.9.1 that should contain some potentially relevant fixes?

[loicmathieu]

Already tried and we have the exact same issue (the stacktrace and core dump are from 0.9.1, I downgrade to 0.9.0 with the same issue)

[carlopi]

The fix in the httpfs extension has gone live a couple of hours ago, could you potentially give another try (while performing before hand FORCE INSTALL httfps once, as explained here: duckdb/duckdb#9340 (comment)?).

This would NOT solve the random_device issue, but might solve the crash if they are independent.

[loicmathieu]

Even with FORCE INSTALL httpfs I have the same issue.

By the way, the random_device issue didn't appears on 0.8.0 so it may not be the same issue as duckdb/duckdb#9340

[Mause]

Can you try the solution mentioned in duckdb/duckdb#8708 (comment) ?

[loicmathieu]

@Mause the solution by setting LD_PRELOAD works, however, this is not a proper fix for us as we cannot control the environment in which our user will run our code.

[Mause]

It's less a permanent solution as it is confirming it's the same issue, and one we've seen before (though that was with tensorflow and python)

[Mause]

For our own reference, are you using any other java libraries that are backed by a C++ library?

[Mause]

I notice /tmp/librocksdbjni14687608028396635175.so is mentioned in the dump, do you see the issue if you exclude that library/don't load it before duckdb?

[loicmathieu]

For our own reference, are you using any other java libraries that are backed by a C++ library?

It is very difficult to answer this question as this kind of information is usually not documented. We are using literally hundreds of libraries (maybe even more than a thousand as we have 400 plugins).

Our runtime uses Netty which, for sure, uses native libraries.

[loicmathieu]

I notice /tmp/librocksdbjni14687608028396635175.so is mentioned in the dump, do you see the issue if you exclude that library/don't load it before duckdb?

Oh!, this can explain why we're only seeing this when using our Kafka runner and not our JDBC runner (we can launch Kestra with two different runners). So yes I confirm this works when we don't use Kafka (so no use of rocksdb).