Skip to content

[Feature Request]: Allow configuring the amount of sharding in DataFrame IOs #24094

Open
@TheNeuralBit

Description

@TheNeuralBit

What would you like to happen?

https://lists.apache.org/thread/9m8mpv9ljdq2h52fdmytvr6ot6sfybbp

Is there any way to control the amount of sharing on the to_parquet call? (I tried adding n_shards = N, but seems to have no effect)

Issue Priority

Priority: 2

Issue Component

Component: dsl-dataframe

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions