Fix incorrect buffer slicing logic in VarChar and VarBinary Accessors - #102
Open
programmer314 wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses a functional bug in how variable-length data is sliced from Arrow buffers within the
SqlAccessorimplementations.Previously, the
getStream(int index)methods inVarCharAccessorandVarBinaryAccessorwere passing the absolute end offset (h.end) as the length parameter to the buffer slice method. In the Netty/Arrow API, the signature isslice(int index, int length).Using the end offset as the length caused the resulting stream to "bleed," including not only the target record but also all subsequent data remaining in the vector's buffer.
Changes
VarCharAccessor.java: UpdatedgetStreamto calculate length ash.end - h.start.VarBinaryAccessor.java: UpdatedgetStreamto calculate length ash.end - h.start.getReader()andgetStream()return the correct byte ranges.Technical Breakdown
When fetching a record from a variable-width vector:
h.start: The byte offset where the value begins.h.end: The byte offset where the value ends.slice(h.start, h.end)incorrectly treats the end-pointer as the count of bytes to read starting fromh.start.slice(h.start, h.end - h.start)ensures the slice is bounded strictly to the bytes belonging to that specific record.Impact
getReader()andgetStream()from returning concatenated data from subsequent rows.