Skip to content

WIP Arrow demo #2338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 9 commits into
base: read_as_arrow
Choose a base branch
from
Draft

WIP Arrow demo #2338

wants to merge 9 commits into from

Conversation

IvoDD
Copy link
Collaborator

@IvoDD IvoDD commented Apr 23, 2025

Reference Issues/PRs

What does this implement or fix?

Any other comments?

Checklist

Checklist for code changes...
  • Have you updated the relevant docstrings, documentation and copyright notice?
  • Is this contribution tested against all ArcticDB's features?
  • Do all exceptions introduced raise appropriate error messages?
  • Are API changes highlighted in the PR description?
  • Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

@IvoDD IvoDD force-pushed the arrow-frontend branch 4 times, most recently from f285e3c to d20a222 Compare April 24, 2025 10:41
@IvoDD IvoDD force-pushed the arrow-frontend branch 2 times, most recently from 4ea5f83 to a57d41a Compare May 1, 2025 14:48
willdealtry and others added 6 commits May 2, 2025 11:10
- Use a shared `ReadResult`
- Return a proper versioned item
With the current implementation of arrow we build a record batch per
data key and hence the index column will be spread over several `MemBlock`s
Removes the slow memory copy workarounds to use proper memory moving to
construct arrow objects.
@IvoDD IvoDD force-pushed the read_as_arrow branch from 94b322f to 2f3d99f Compare May 2, 2025 13:18
@IvoDD IvoDD force-pushed the read_as_arrow branch from 2f3d99f to 2b8f58d Compare May 2, 2025 13:21
@IvoDD IvoDD force-pushed the arrow-frontend branch from a57d41a to b1f80f4 Compare May 5, 2025 11:15
@IvoDD IvoDD force-pushed the arrow-frontend branch from b1f80f4 to 16f3447 Compare May 5, 2025 11:28
@IvoDD IvoDD force-pushed the read_as_arrow branch 5 times, most recently from 03550bd to 9274a4c Compare May 23, 2025 13:50
@IvoDD IvoDD force-pushed the read_as_arrow branch 6 times, most recently from bed5dad to df1372b Compare May 27, 2025 11:55
@IvoDD IvoDD force-pushed the read_as_arrow branch 4 times, most recently from 5197f9b to 9274a4c Compare May 28, 2025 07:03
@IvoDD IvoDD force-pushed the read_as_arrow branch 6 times, most recently from 65d4e59 to bef748c Compare June 6, 2025 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants