-
Notifications
You must be signed in to change notification settings - Fork 195
Extend array rapidcheck templates to support nullable and variable-length fields #5511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…-templates-var-validity
…-templates-var-validity
ypatia
approved these changes
Jun 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the detailed description, that helped a lot understanding what's in here.
Co-authored-by: Ypatia Tsavliri <[email protected]>
Co-authored-by: Ypatia Tsavliri <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves CORE-23.
#5417 added a suite of tests which create an array with random data, and then read back the data and compare against the input.
Those tests were limited to single-valued non-nullable attributes. This pull request extends their capabilities to support nullable attributes and variable-length attributes (and nullable variable-length attributes).
Previously the code all used
std::vector<T>
to represent the single-valued attribute data. Now to represent the set of buffers needed to write or read a user type, we usestruct query_buffers<T>
whereT
some logical representation of the user type:T
for a non-nullable single-valued cellstd::vector<T>
for a fundamental typeT
for a non-nullable variable-length cellstd::optional<T>
for a fundamental typeT
for a nullable single-valued cellstd::optional<std::vector<T>>
for a fundamental typeT
for a nullable variable-length cellThe specializations of
query_buffers
contain the columnar buffers for each of these.The
run_execute
function which drives the queries does multi-part reads into a largequery_buffers
which it then compares against the expected result. To handle the variable-length data it is necessary to update theuint64_t outcursor
which previously just held the number of cells already read to a tuple which contains the variable-length position as well as the cell position.Because this is done with static dispatch there's a lot of
std::apply
which basically means "turn this tuple into a variadic function argument list".Testing
One of the existing tests is updated to use a non-nullable variable-length attribute and one is updated to use a nullable variable-length attribute.
TYPE: NO_HISTORY
DESC: Enable nullable/variable data in array rapidcheck tests