-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reuse CRC when appending index components #1272
base: main
Are you sure you want to change the base?
Conversation
a7fdb55
to
4697f26
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have unit tests that cover this change?
Yes, there is. |
4697f26
to
8ba1820
Compare
Ok, apparently our usage is a bit more complex. |
src/java/org/apache/cassandra/index/sai/utils/IndexFileUtils.java
Outdated
Show resolved
Hide resolved
8ba1820
to
4b8aae8
Compare
@eolivelli can you have another look? I hardened it. It's not perfect but I can only go so far in Java without affine types ;) |
4b8aae8
to
5915713
Compare
src/java/org/apache/cassandra/index/sai/disk/format/IndexDescriptor.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that the current approach works, but I would feel more confident if we add some unit tests about concurrent access.
src/java/org/apache/cassandra/index/sai/utils/IndexFileUtils.java
Outdated
Show resolved
Hide resolved
src/java/org/apache/cassandra/index/sai/utils/IndexFileUtils.java
Outdated
Show resolved
Hide resolved
src/java/org/apache/cassandra/index/sai/utils/IndexFileUtils.java
Outdated
Show resolved
Hide resolved
48ffd81
to
f506fc9
Compare
There are plenty of scenarios tested in |
The class is not thread safe, but we are adding some shared state, so it is not a matter of testing a single instance, but multiple instances accessing the same entries in the shared state |
No, the added state is still split by the file name; it is not shared. The only thing that's shared is the cache instance, but that one is thread safe, and I guess it's well tested and does the right thing. Should I test library code? Earlier: Now: What particular scenario do you have in mind? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, can you create corresponding CNDB pr and make sure the multi-segment index tests in SingleCompactorTest
passes?
This commit improves performance of appending SAI components by avoiding unnecessary computation of CRC from the beginning of the file each time it is opened for appending. Fixes riptano/cndb#10783
5bed2c7
to
f4a176c
Compare
Quality Gate passedIssues Measures |
I added warnings and I created a PR on the cndb side to test @jasonstack is it good now? Can I merge? |
Fixes #10873