Skip to content

pack_partitions_to_parquet hangs silently if destination already exists and overwrite=False #181

Open
@ahnsws

Description

@ahnsws

Hello, it looks like if I write to a parquet folder using pack_partitions_to_parquet, it runs successfully once, but upon running it again, if overwrite is False, it silently hangs. The script is below:

import dask.dataframe as dd
import spatialpandas as spd
from spatialpandas.dask import DaskGeoDataFrame

points = spd.geometry.PointArray([[0, 0], [1, 1]])
sdf = spd.GeoDataFrame({"geometry": points})
sddf = dd.from_pandas(sdf, npartitions=2)
assert isinstance(sddf, DaskGeoDataFrame)

sddf.pack_partitions_to_parquet("data.parq")
sddf.pack_partitions_to_parquet("data.parq")  # hangs silently

My environment is:

OS: Ubuntu 22.04.5 LTS
python 3.11
dask 2025.3
distributed 2025.3
spatialpandas 0.5.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions