How BEGIN TRANSACTION; COMMIT; creates parquet files in DuckLake? #256
-
If I wrap a number of insert statements between BEGIN TRANSACTION; COMMIT; statements does DuckLake create one parquet file, rather than several files if the inserts were not grouped in a transaction? Or should one not worry and use merge_adjacent_files with ducklake_cleanup_old_files and let DuckLake manage files? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
No, parquet files are written optimistically on INSERT. If you are worried about small files you should indeed rely on |
Beta Was this translation helpful? Give feedback.
No, parquet files are written optimistically on INSERT.
They're automatically cleaned up on abort of the transaction.
If you are worried about small files you should indeed rely on
merge_adjacent_files
or make use of data-inlining