feat: add blob take and BlobFile read APIs - #82
Open
MisterRaindrop wants to merge 5 commits into
Open
Conversation
Adds LanceBlobHandling and lance_scanner_set_blob_handling(), applied in build_scanner() right after project(). Values outside the enum and calls after the scan has started are rejected.
Two-fragment Blob v2 fixture (inline, packed, dedicated, empty and null rows) and six tests. The C and C++ smoke programs get a blob dataset as a third argument and check the setter too.
lance_dataset_take_blobs / _by_indices fill a caller array of LanceBlobFile* (NULL for null values, untouched on error). The handle has size, read, read_up_to, read_range, seek, tell and close, and wraps upstream BlobFile only, so it outlives the dataset.
Reuses the fixture from the scanner blob handling PR, with a stable row id switch.
MisterRaindrop
force-pushed
the
feat/blob-take-api
branch
from
September 10, 2026 10:22
182df3b to
96656bc
Compare
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The opaque LanceBlobFile wrapper closes the C-access gap documented in #76 while preserving upstream random and range reads instead of forcing large payloads into scan batches. Input validation happens before caller output is touched, handle ownership is independent of the dataset, and the multi-fragment coverage exercises inline, packed, dedicated, empty, null, bounds, and cursor behavior through Rust, C, and C++.
This is stacked on #81, whose current head exactly matches this branch’s first two commits; land #81 first, then this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #81; the first two commits are that PR.
Adds the row-addressed blob API from #76:
lance_dataset_take_blobs(by_rowid) andlance_dataset_take_blobs_by_indices(by row offset) fill a caller array ofLanceBlobFile*, NULL for a null value; the handle hassize,read,read_up_to,read_range,seek,tellandclose. Reads copy into caller buffers and return 0 / -1 with the thread-local error set.The handle wraps upstream
BlobFileonly, so it outlives the dataset. Take is all-or-nothing (outuntouched on error),readrejects a buffer smaller than the remaining bytes instead of truncating, and an unknown column is an invalid argument rather thanLANCE_ERR_INTERNAL.lance.hppgets an RAIIBlobFileandDataset::take_blobs*()returningstd::vector<std::optional<BlobFile>>.Tests: sixteen
test_blob_*cases (both entry points over every storage layout, with and without stable row ids, cursor semantics, bounds and overflow, handles outliving the dataset, NULL and invalid arguments withoutuntouched) plus the C and C++ smoke programs.Open:
readgoes from the cursor to the end like the Rust and Javaread(), andseekis not bounds-checked, like upstream.take_blobs_by_addressesandkind/uriaccessors are not included.