Open
Description
dlt version
1.10.0
Describe the problem
This change removed the loader_file_format
from the NormalizeConfiguration
class.
According to the docs (e.g. here), you should be able to specificy loader_file_format="parquet"
in the [normalize]
section, and dlt would then write Parquet files to the destination. But this option doesn't work anymore now.
Expected behavior
The loader file format should be according to what's in .dlt/config.toml
.
Steps to reproduce
Add this to your config.toml
and see how the result is always written as jsonl
after running a pipeline:
[normalize]
loader_file_format="parquet"
Operating system
Linux
Runtime environment
Local
Python version
3.13
dlt data source
Filesystem, SQL, REST API
dlt destination
No response
Other deployment details
No response
Additional information
No response
Metadata
Metadata
Assignees
Type
Projects
Status
In Progress