Skip to content

Extend array rapidcheck templates to support nullable and variable-length fields #5511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jun 11, 2025

Conversation

rroelke
Copy link
Member

@rroelke rroelke commented May 13, 2025

Resolves CORE-23.

#5417 added a suite of tests which create an array with random data, and then read back the data and compare against the input.

Those tests were limited to single-valued non-nullable attributes. This pull request extends their capabilities to support nullable attributes and variable-length attributes (and nullable variable-length attributes).

Previously the code all used std::vector<T> to represent the single-valued attribute data. Now to represent the set of buffers needed to write or read a user type, we use struct query_buffers<T> where T some logical representation of the user type:

  • some fundamental type T for a non-nullable single-valued cell
  • std::vector<T> for a fundamental type T for a non-nullable variable-length cell
  • std::optional<T> for a fundamental type T for a nullable single-valued cell
  • std::optional<std::vector<T>> for a fundamental type T for a nullable variable-length cell

The specializations of query_buffers contain the columnar buffers for each of these.

The run_execute function which drives the queries does multi-part reads into a large query_buffers which it then compares against the expected result. To handle the variable-length data it is necessary to update the uint64_t outcursor which previously just held the number of cells already read to a tuple which contains the variable-length position as well as the cell position.

Because this is done with static dispatch there's a lot of std::apply which basically means "turn this tuple into a variadic function argument list".

Testing

One of the existing tests is updated to use a non-nullable variable-length attribute and one is updated to use a nullable variable-length attribute.


TYPE: NO_HISTORY
DESC: Enable nullable/variable data in array rapidcheck tests

@rroelke rroelke requested a review from ypatia June 6, 2025 19:17
Copy link
Member

@ypatia ypatia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the detailed description, that helped a lot understanding what's in here.

@rroelke rroelke merged commit 4fa5306 into main Jun 11, 2025
56 checks passed
@rroelke rroelke deleted the rr/CORE-23-rapidcheck-templates-var-validity branch June 11, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants