Skip to content

[FEA] explore dropping arrow dependency #2347

Open
@revans2

Description

@revans2

Is your feature request related to a problem? Please describe.
CUDF is in the middle of trying to drop arrow as a dependency. At least as a public dependency. A lot of this is to maintain compatibility in a python ecosystem where arrow could be installed on a system separate from CUDF.

We don't really have as much of the same issue because of how we link things, but also because of java vs python. Still it would be good for us to stop using arrow directly in spark-rapids-jni. We currently only use it for parsing parquet footers, but that is code we could lift from arrow an maintain ourselves, as it really is just the thrift generated classes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions