FATAL Error: Failed to create checkpoint because of error...

[ajweber]

JAVA 17, DuckDB 1.2 (duckdb_jdbc-1.2.1.jar)

I am trying to perform some queued INSERTs (specifically preparedStatement, batch inserts) using CompletableFuture (i.e. background executor, but same process). All these go to the same table.

The background method uses (partially) :
try (var connection = duckDBConnection.duplicate(); var statement = connection.prepareStatement(INSERT_TEMPLATE))) { for (...) { statement.setString(); statement.set... statement.addBatch(); ... } statement.executeBatch(); }

I'm using CompletableFuture.runAsync, so these go on the Java commonPool() executor, queued to run asap.

After a few of these complete, I'm getting the exception:
java.sql.SQLException: FATAL Error: Failed to create checkpoint because of error: Failed to delete file "e:\vectorindex\efr.duckdb.wal": The process cannot access the file because it is being used by another process.

at org.duckdb.DuckDBNative.duckdb_jdbc_execute(Native Method)
at org.duckdb.DuckDBPreparedStatement.execute(DuckDBPreparedStatement.java:148)
at org.duckdb.DuckDBPreparedStatement.executeBatchedPreparedStatement(DuckDBPreparedStatement.java:489)
at org.duckdb.DuckDBPreparedStatement.executeBatch(DuckDBPreparedStatement.java:474)
...`

This is entirely unexpected since all of the code is running in the same process, and according to the docs, this should be fine. And they are all unique INSERTs (there isn't even a PK on the table), so there should be no row-based table contention.

It appears that DuckDB is performing checkpoints on its own...maybe when the duplicate session is closing in the method? And whether it is performing checkpoints or not, this appears to be a bug, because it's documented that multiple threads within the same process should be able to r/w the same database.

Does the WAL strategy need to be adjusted (internally)?

Are there any workarounds for now?

Thanks!

[ajweber]

Apologies for the poor formatting of my OP. I originally had it posted in the rust sub repo erroneously and tried to copy-and-paste it here with limited effect.