Skip to content

[FEA] explore dropping arrow dependency #2347

Open
@revans2

Description

@revans2

Is your feature request related to a problem? Please describe.
CUDF is in the middle of trying to drop arrow as a dependency. At least as a public dependency. A lot of this is to maintain compatibility in a python ecosystem where arrow could be installed on a system separate from CUDF.

We don't really have as much of the same issue because of how we link things, but also because of java vs python. Still it would be good for us to stop using arrow directly in spark-rapids-jni. We currently only use it for parsing parquet footers, but that is code we could lift from arrow an maintain ourselves, as it really is just the thrift generated classes.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions