Skip to content

feat: support table partitioning policies - #596

Open
sshaplygin wants to merge 1 commit into
ydb-platform:masterfrom
sshaplygin:feat/table-partitioning-policies
Open

feat: support table partitioning policies#596
sshaplygin wants to merge 1 commit into
ydb-platform:masterfrom
sshaplygin:feat/table-partitioning-policies

Conversation

@sshaplygin

Copy link
Copy Markdown
Contributor

Closes #311.

CreateTableRequest only carried path, columns, primary key and attributes, so partitioning could not be configured from the Rust SDK at all. This adds the table-side equivalent of ydb-go-sdk's options.WithPartitioningSettings, WithUniformPartitions and WithExplicitPartitions.

API

table_client.create_table(
    CreateTableRequest::new(path)
        .with_column(TableColumn::new("id", Value::Uint64(0)))
        .with_primary_key(["id"])
        .with_partitioning_settings(
            TablePartitioningSettings::new()
                .with_partitioning_by_size(true)
                .with_partition_size_mb(128)
                .with_min_partitions_count(2)
                .with_max_partitions_count(16),
        )
        .with_uniform_partitions(4),
).await?;
Added Purpose
TablePartitioningSettings Auto-partitioning policy and partition-count bounds
CreateTableRequest::with_partitioning_settings Policy for a new table
CreateTableRequest::with_uniform_partitions / with_partition_at_keys Initial partition layout (the partitions oneof)
AlterTableRequest::alter_partitioning_settings Change the policy of an existing table
TableDescription::partitioning_settings Read the policy back

Every settings field is Option, mirroring the tri-state feature flags on the wire: unset means "server default" on create and "leave alone" on alter. Zero counts coming back from the server decode as None, so a describe result round-trips into an alter request unchanged.

Two things worth reviewing

Naming. The types are TablePartitioningSettings and TablePartitions, not the unprefixed names. The crate re-exports everything flat from the root, and PartitioningSettings is already the topic client's type — Go avoids the clash with separate packages, which is not available here. Happy to rename if you prefer a different convention.

Explicit split points needed the server to pin down. A split point is a prefix of the primary key, so YDB expects Tuple<Optional<T>, ...>. Sending a bare scalar is rejected with Partition ranges are not sorted at index 0 even when the values are strictly ascending — which is what my first attempt did. The public Value has no tuple variant yet (#309), so with_partition_at_keys takes one Vec<Value> per split point and builds the tuple in the raw layer:

.with_partition_at_keys([vec![Value::Uint64(100)], vec![Value::Uint64(200)]])

If #309 lands a Value::Tuple, this can be simplified without changing the signature.

Verification

Against ydbplatform/local-ydb:nightly, the image CI uses. Three integration tests, all exercising a real server:

  • settings survive create → describe, and alter → describe;
  • with_uniform_partitions applies;
  • explicit split points place rows across partitions (checked with bulk_upsert + read_rows), and descending split points are rejected.

Unit tests cover the feature-flag tri-state and the zero-count decoding.

cargo test --workspace -- --include-ignored   # 309 passed, 0 failed
cargo test -p ydb --doc                       # 32 passed
cargo fmt --check
cargo clippy --workspace --all-targets --no-deps --exclude=ydb-grpc -- -D warnings

Not included

The legacy TableProfile.PartitioningPolicy message (preset names, AUTO_SPLIT / AUTO_SPLIT_MERGE) is untouched — PartitioningSettings is the modern replacement and the one the Go SDK's documented options map onto. Say the word if parity there is wanted too.

🤖 Generated with Claude Code

Closes ydb-platform#311.

`CreateTableRequest` could only set path, columns, primary key and
attributes, so there was no way to configure partitioning from the Rust
SDK. Adds the table-side equivalent of ydb-go-sdk's
`options.WithPartitioningSettings` / `WithUniformPartitions` /
`WithExplicitPartitions`.

- `TablePartitioningSettings` with chained setters for `partitioning_by`,
  `partitioning_by_size`, `partition_size_mb`, `partitioning_by_load`,
  `min_partitions_count` and `max_partitions_count`. Every field is
  optional, mirroring the tri-state feature flags on the wire: unset
  means "server default" on create and "leave alone" on alter.
- `CreateTableRequest::with_partitioning_settings`,
  `with_uniform_partitions` and `with_partition_at_keys`.
- `AlterTableRequest::alter_partitioning_settings`.
- `TableDescription::partitioning_settings`, so a policy can be read back.

The types are named `TablePartitioningSettings` / `TablePartitions`
rather than dropping the prefix: the crate re-exports everything flat
from the root and `PartitioningSettings` is already taken by the topic
client. Go keeps them apart by package instead.

Explicit split points needed the server to disambiguate them. A split
point is a *prefix of the primary key*, so YDB expects
`Tuple<Optional<T>, ...>`; sending a bare scalar is rejected with
"Partition ranges are not sorted at index 0" even when the values are
ascending. The public `Value` has no tuple variant yet (ydb-platform#309), so
`with_partition_at_keys` takes one `Vec<Value>` per split point and
builds the tuple in the raw layer.

Verified against ydbplatform/local-ydb:nightly: settings survive
create -> describe and alter -> describe, uniform partitioning applies,
explicit split points place rows across partitions, and descending split
points are rejected. 309 tests pass with --include-ignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.66667% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.05%. Comparing base (a6d7911) to head (726a4a3).

Files with missing lines Patch % Lines
ydb/src/table_requests.rs 90.56% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #596      +/-   ##
==========================================
+ Coverage   86.91%   87.05%   +0.14%     
==========================================
  Files         198      199       +1     
  Lines       19492    19788     +296     
==========================================
+ Hits        16941    17227     +286     
- Misses       2551     2561      +10     
Flag Coverage Δ
rust-1.88.0 87.05% <96.66%> (+0.14%) ⬆️
rust-1.96.1 87.30% <96.66%> (+0.14%) ⬆️
tests 87.05% <96.66%> (+0.14%) ⬆️
ubuntu 87.05% <96.66%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Partition policies

1 participant