What is the problem the feature request solves?
native_iceberg_compat reads parquet in Rust via NativeBatchReader, then imports the data to JVM via Arrow FFI, then re-exports it back to native for execution by the next operator in the plan. Although Arrow FFI is zero-copy for the data, there is still serialization overhead for the schema.
I am wondering if this round-trip can be avoided in some cases
The following profiling information is from running async-profiler.
Describe the potential solution
No response
Additional context
No response
What is the problem the feature request solves?
native_iceberg_compatreads parquet in Rust viaNativeBatchReader, then imports the data to JVM via Arrow FFI, then re-exports it back to native for execution by the next operator in the plan. Although Arrow FFI is zero-copy for the data, there is still serialization overhead for the schema.I am wondering if this round-trip can be avoided in some cases
The following profiling information is from running async-profiler.
Describe the potential solution
No response
Additional context
No response