Open
Description
Environment
deltalake: 0.25.5
polars: 1.29.0
pyarrow: 20.0.0
Binding: Python
Bug
When writing a Polars DataFrame that has a column of type List(String) and a row value of an empty list, I see a memory alignment error.
The following code reproduces it:
from deltalake import write_deltalake
import polars as pl
table = pl.DataFrame(data={"col1": [[]]}, schema=[("col1", pl.List(pl.String()))])
write_deltalake("test_table", table, mode="append")
Error message:
_internal.DeltaError: Generic DeltaTable error: Join Error caused by External error: task 30 panicked with message "Memory pointer from external source (e.g, FFI) is not aligned with the specified scalar type. Before importing buffer through FFI, please make sure the allocation is aligned.