Skip to content

DSP-25176 - #2544

Open
blambov wants to merge 5 commits into
mainfrom
DSP-25176
Open

DSP-25176#2544
blambov wants to merge 5 commits into
mainfrom
DSP-25176

Conversation

@blambov

@blambov blambov commented Jul 28, 2026

Copy link
Copy Markdown

What is the issue

DSP-25176

Encrypted primary index files are written in a way that hides encryption metadata in holes in the file. Non-trie data like primary keys may span over these holes, which is handled correctly when we read these keys.
However, when we skip a key instead of reading it, we may end up in the wrong position if the key is longer than the metadata.

What does this PR fix and why was it fixed

Implements skip position correction for encrypted primary index files.
Also corrects problems listing partitions when the boundaries of an sstable have been reduced due to zero-copy streaming metadata.
Adds tests for both problems.
Disables early opening for online scrub, which is unsafe because data may transiently disappear.
Makes sure that failed attempts to open a compaction result early are logged but do not stop the operation, by rolling back the prepared transaction checkpoint on error.
Fixes incorrect row index length in openFinalEarly.
Fixes incorrect DecoratedKey minimization.

CC5 PR: #2547
DSE PR: https://github.com/riptano/bdp/pull/21750

@github-actions

Copy link
Copy Markdown

Checklist before you submit for review

  • This PR adheres to the Definition of Done
  • Make sure there is a PR and ticket in the CNDB project updating the Converged Cassandra version
  • Use NoSpamLogger for log lines that may appear frequently in the logs
  • Verify test results on Butler
  • Test coverage for new/modified code is > 80%, check manually at SonarCloud page
  • Proper code formatting
  • Proper title for each commit staring with the project-issue number, like CNDB-1234
  • Each commit has a meaningful description
  • Each commit is not very long and contains related changes
  • Renames, moves and reformatting are in distinct commits
  • All new files should contain the IBM copyright header instead of the Apache License one (no DataStax copyright any longer)

@blambov
blambov requested a review from bereng July 28, 2026 15:51
@bereng

bereng commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

@blambov This code looks very familiar to the DSE side one. The only CI failure Butler reports it's a timeout.

I only have one q: Does the last commit lack a test maybe?

@blambov

blambov commented Aug 5, 2026

Copy link
Copy Markdown
Author

Test added.

blambov and others added 5 commits August 11, 2026 14:31
Encrypted primary index files are written in a way that hides
encryption metadata in holes in the file. Non-trie data like primary
keys may span over these holes, which is handled correctly when we
read these keys. However, when we skip a key instead of reading it,
we may end up in the wrong position if the key is longer than the
metadata.
Log error and let operations continue when an early open attempt fails.
Change online scrub to not use early open as it is unsafe.
@sonarqube-dx-prod

Copy link
Copy Markdown

Quality Gate failed Quality Gate failed

Failed conditions
2 New issues

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE SonarQube for IDE

@plpesvc-ds

Copy link
Copy Markdown

❌ Build ds-cassandra-pr-gate/PR-2544 rejected by Butler


3 regressions found
See build details here


Found 3 new test failures

Test Explanation Runs Upstream
o.a.c.index.sai.cql.VectorCompaction100dTest.testPQRefine[version=ed enableNVQ=false] REGRESSION 🔴 0 / 30
o.a.c.index.sai.cql.VectorSiftSmallTest.testSiftSmall[version=fb enableNVQ=false enableFused=false] REGRESSION 🔴 0 / 30
o.a.c.index.sai.cql.VectorTypeTest.lwtTest[version=db enableNVQ=false enableFused=false] REGRESSION 🔴 0 / 30

Found 7 known test failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants