-
Notifications
You must be signed in to change notification settings - Fork 21
Add arrow array stream #608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #608 +/- ##
==========================================
- Coverage 88.45% 88.10% -0.36%
==========================================
Files 104 108 +4
Lines 8041 8286 +245
==========================================
+ Hits 7113 7300 +187
- Misses 928 986 +58
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
0ba41f3 to
9d697cc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements support for the Arrow C Stream Interface specification, enabling streaming data exchange between different libraries within a single process. The implementation provides both low-level C interface callbacks and a high-level C++ proxy class for managing ArrowArrayStream objects.
Key changes:
- Added
arrow_array_stream_proxyclass for RAII-based stream management with push/pop operations - Implemented all mandatory ArrowArrayStream callbacks (get_schema, get_next, get_last_error, release)
- Added
check_compatible_schemafunction to validate schema compatibility when adding arrays to streams
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| include/sparrow/c_stream_interface.hpp | Defines the ArrowArrayStream C structure per the Arrow C Stream Interface specification |
| include/sparrow/arrow_interface/arrow_array_stream.hpp | Declares stream callback functions and helper functions |
| include/sparrow/arrow_interface/arrow_array_stream/private_data.hpp | Implements private data class managing schema and array queue for streams |
| include/sparrow/arrow_interface/arrow_array_stream_proxy.hpp | Provides C++ proxy class with templated push/pop operations |
| src/arrow_interface/arrow_array_stream.cpp | Implements stream callback functions |
| src/arrow_interface/arrow_array_stream_proxy.cpp | Implements proxy class methods including lifecycle management |
| include/sparrow/arrow_interface/arrow_schema.hpp | Declares new check_compatible_schema function and removes unused include |
| src/arrow_interface/arrow_schema.cpp | Implements schema comparison logic with recursive checking and explicit null assignments |
| test/test_arrow_array_stream_proxy.cpp | Comprehensive test suite covering stream operations and edge cases |
| test/test_arrow_schema.cpp | Tests for check_compatible_schema covering various mismatch scenarios |
| test/CMakeLists.txt | Adds new test file to build configuration |
| CMakeLists.txt | Adds new source and header files, reorganizes file order alphabetically |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d007e8a to
9d697cc
Compare
No description provided.