CHECKPOINT fails with Azure Blob Storage (SSL CA cert error)

[rec00n]

What happens?

I'am testing DuckLake with Postgres and Azure Blob Storage as storage backend. When I wanted to run the CHECKPOINT; statement I am running into:

"Fail to get a new connection for: https://myaccount.blob.core.windows.net/. Problem with the SSL CA cert (path? access rights?)"

To Reproduce

It tried to debug it, but strangely, if I run ...

CALL ducklake_flush_inlined_data('bla'); CALL ducklake_merge_adjacent_files('bla'); CALL ducklake_expire_snapshots('bla', older_than => now() - INTERVAL '10 minutes'); CALL ducklake_cleanup_old_files('bla',cleanup_all => true);

...everything works fine as expected and I can see parquet files created/modified in the blob storage container.

I already set also SET azure_transport_option_type = 'curl';, so I don't think the issue is here.

The code is running in a python:3.13-trixie container and ca-certificates is present.

Generally is not a big deal, since you can execute the maintenance statements as desired, but probably is something which should be investigated.

OS:

python:3.13-trixie

DuckDB Version:

1.4.4

DuckLake Version:

3f1b372

DuckDB Client:

Python

Hardware:

No response

Full Name:

Bernhard Castellazzi

Affiliation:

Salzburg

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have

[rec00n]

After individually calling all CHECKPOINT; maintenance functions I could pinpoint that ducklake_delete_orphaned_files is causing this error. All other functions run without problem.

[rec00n]

#766

[guillesd]

Hi @rec00n is this working on 1.5?

[rec00n]

@guillesd
Hey, unfortunately I am still seeing the same error when running ducklake_delete_orphaned_files with DuckDB 1.5.0:

CALL ducklake_delete_orphaned_files('myducklake', cleanup_all => true);', error: Failed to get files scheduled for deletion from DuckLake: Fail to get a new connection for: https://*****.dfs.core.windows.net. Problem with the SSL CA cert (path? access rights?)

All other functions seem to work fine.

[guillesd]

Gonna try to get it checked internally (I am not very involved with the Azure extension).

[rec00n]

Hi, fyi, still not works with 1.5.2 ...

_duckdb.Error: Failed to get files scheduled for deletion from DuckLake: Failed to get token from ChainedTokenCredential. │ │ │ │ LINE 2: FROM read_blob('abfss://mydata/' || '**')

[rec00n]

Hi @guillesd, is there any update on this issue? Or would it make more sense to build some kind of custom workaround until it is solved?

[rec00n]

If anyone is still struggling with that problem: What solved it in my case was setting SET GLOBAL azure_transport_option_type = 'curl';

I don't know if that points to a design problem, or simply should be documented somewhere.