Skip to content

[FEA] Python bindings to pack/unpack #7601

Closed
@jakirkham

Description

@jakirkham

Is your feature request related to a problem? Please describe.

When shipping DataFrames over the wire or spilling them, it can be handy to pack them into a more compact single buffer first and then unpack them into multiple buffers at the other end.

Describe the solution you'd like

Recently this functionality was added at the C++ layer ( #7096 ). It would be good to have bindings to this for Python and use this in relevant serialize/deserialize methods.

Describe alternatives you've considered

We could do this packing elsewhere like in Distributed ( dask/distributed#3732 ). Though this would then not use the C++ implementation here. It also wouldn't solve this for other Python use cases

Additional context

We've discussed adding this to a config potentially ( #5311 ). Not sure if this is still needed given the newer C++ implementation

Metadata

Metadata

Assignees

Labels

PythonAffects Python cuDF API.feature requestNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions