Parquet import - Schema / DDL derive

[fridolin-koch]

We are currently looking into Ducklake for our archive system, doing some kind of feasibility analysis. We already have tooling to archive systems to (partitioned) parquet files.

As far as I understand, I could add files using ducklake_add_data_files. But I have to create the Schema/DDL for the table before doing that?

Given a folder structure like table_name/*.parquet (or alternatively also hive-style part.).

Is it possible to somehow automatically create the table schema based on the parquet files? If not is something like this planned in the future or should we rather implement this as part of our extraction process (i.e. generating the create table ddl)?

Thanks in advance!

[J-Meyers]

You can go this approach to create the table schemas without actually doing an insert, as just one extra command

CREATE TABLE my_ducklake.table_name AS (FROM 'table_name/*.parquet' LIMIT 0);