feat: support table partitioning policies - #596
Open
sshaplygin wants to merge 1 commit into
Open
Conversation
Closes ydb-platform#311. `CreateTableRequest` could only set path, columns, primary key and attributes, so there was no way to configure partitioning from the Rust SDK. Adds the table-side equivalent of ydb-go-sdk's `options.WithPartitioningSettings` / `WithUniformPartitions` / `WithExplicitPartitions`. - `TablePartitioningSettings` with chained setters for `partitioning_by`, `partitioning_by_size`, `partition_size_mb`, `partitioning_by_load`, `min_partitions_count` and `max_partitions_count`. Every field is optional, mirroring the tri-state feature flags on the wire: unset means "server default" on create and "leave alone" on alter. - `CreateTableRequest::with_partitioning_settings`, `with_uniform_partitions` and `with_partition_at_keys`. - `AlterTableRequest::alter_partitioning_settings`. - `TableDescription::partitioning_settings`, so a policy can be read back. The types are named `TablePartitioningSettings` / `TablePartitions` rather than dropping the prefix: the crate re-exports everything flat from the root and `PartitioningSettings` is already taken by the topic client. Go keeps them apart by package instead. Explicit split points needed the server to disambiguate them. A split point is a *prefix of the primary key*, so YDB expects `Tuple<Optional<T>, ...>`; sending a bare scalar is rejected with "Partition ranges are not sorted at index 0" even when the values are ascending. The public `Value` has no tuple variant yet (ydb-platform#309), so `with_partition_at_keys` takes one `Vec<Value>` per split point and builds the tuple in the raw layer. Verified against ydbplatform/local-ydb:nightly: settings survive create -> describe and alter -> describe, uniform partitioning applies, explicit split points place rows across partitions, and descending split points are rejected. 309 tests pass with --include-ignored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #596 +/- ##
==========================================
+ Coverage 86.91% 87.05% +0.14%
==========================================
Files 198 199 +1
Lines 19492 19788 +296
==========================================
+ Hits 16941 17227 +286
- Misses 2551 2561 +10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #311.
CreateTableRequestonly carried path, columns, primary key and attributes, so partitioning could not be configured from the Rust SDK at all. This adds the table-side equivalent of ydb-go-sdk'soptions.WithPartitioningSettings,WithUniformPartitionsandWithExplicitPartitions.API
TablePartitioningSettingsCreateTableRequest::with_partitioning_settingsCreateTableRequest::with_uniform_partitions/with_partition_at_keyspartitionsoneof)AlterTableRequest::alter_partitioning_settingsTableDescription::partitioning_settingsEvery settings field is
Option, mirroring the tri-state feature flags on the wire: unset means "server default" on create and "leave alone" on alter. Zero counts coming back from the server decode asNone, so a describe result round-trips into an alter request unchanged.Two things worth reviewing
Naming. The types are
TablePartitioningSettingsandTablePartitions, not the unprefixed names. The crate re-exports everything flat from the root, andPartitioningSettingsis already the topic client's type — Go avoids the clash with separate packages, which is not available here. Happy to rename if you prefer a different convention.Explicit split points needed the server to pin down. A split point is a prefix of the primary key, so YDB expects
Tuple<Optional<T>, ...>. Sending a bare scalar is rejected withPartition ranges are not sorted at index 0even when the values are strictly ascending — which is what my first attempt did. The publicValuehas no tuple variant yet (#309), sowith_partition_at_keystakes oneVec<Value>per split point and builds the tuple in the raw layer:If #309 lands a
Value::Tuple, this can be simplified without changing the signature.Verification
Against
ydbplatform/local-ydb:nightly, the image CI uses. Three integration tests, all exercising a real server:with_uniform_partitionsapplies;bulk_upsert+read_rows), and descending split points are rejected.Unit tests cover the feature-flag tri-state and the zero-count decoding.
Not included
The legacy
TableProfile.PartitioningPolicymessage (preset names,AUTO_SPLIT/AUTO_SPLIT_MERGE) is untouched —PartitioningSettingsis the modern replacement and the one the Go SDK's documented options map onto. Say the word if parity there is wanted too.🤖 Generated with Claude Code