Ducklake persistent cache #226
Description
Describe the bug
Ducklake S3 requests are not cached.
To Reproduce
If I try to preload cache_httpfs I get
load cache_httpfs;
load ducklake;
create secret ....
attach [my_ducklake_with_postgres_meta_and_s3_data]
Missing Extension Error:
Data path s3://[my_s3_path] requires the extension httpfs to be loaded
Please try installing and loading the httpfs extension by running:
INSTALL httpfs;
LOAD httpfs;
On the other hand if I do
load httfs;
load cache_httpfs;
load ducklake;
....
I don't get any cache hits reported from e.g.
SELECT * FROM cache_httpfs_cache_access_info_query();
which outputs
┌─────────────┬─────────────────┬──────────────────┐
│ cache_type │ cache_hit_count │ cache_miss_count │
│ varchar │ uint64 │ uint64 │
├─────────────┼─────────────────┼──────────────────┤
│ metadata │ 0 │ 0 │
│ data │ 0 │ 0 │
│ file handle │ 0 │ 0 │
│ glob │ 0 │ 0 │
└─────────────┴─────────────────┴──────────────────┘
after multiple executions of the same query.
I have also tried with
SET enable_external_file_cache=true;
Expected behavior
Cache hits after multiple queries.
Screenshots
Desktop (please complete the following information):
- OS: Linux
- duckdb v1.3.2
┌──────────────────┬──────────────┬─────────────────────┬──────────────────┬─────────────────┐
│ extension_name │ repository │ update_result │ previous_version │ current_version │
│ varchar │ varchar │ varchar │ varchar │ varchar │
├──────────────────┼──────────────┼─────────────────────┼──────────────────┼─────────────────┤
│ cache_httpfs │ community │ NO_UPDATE_AVAILABLE │ f309871 │ f309871 │
│ postgres_scanner │ core │ NO_UPDATE_AVAILABLE │ c0411b9 │ c0411b9 │
│ ducklake │ core_nightly │ NO_UPDATE_AVAILABLE │ 6a82469 │ 6a82469 │
│ httpfs │ core │ NO_UPDATE_AVAILABLE │ af7bcaf │ af7bcaf │