GH-45371: [C++] Fix data race in SimpleRecordBatch::columns
#45372
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
mapleFU
reviewed
Jan 29, 2025
mapleFU
reviewed
Jan 29, 2025
cpp/src/arrow/record_batch_test.cc
Outdated
| auto schema = ::arrow::schema({field("f1", utf8())}); | ||
| auto record_batch = RecordBatch::Make(schema, length, {array_data}); | ||
| std::atomic_bool start_flag{false}; | ||
| std::thread t([record_batch, &start_flag]() { |
There was a problem hiding this comment.
should more than one thread be tested here?
There was a problem hiding this comment.
Yes, this should use several threads that would do the same thing concurrently.
There was a problem hiding this comment.
In the current test the race is between t and the main thread. Only 2 threads are necessary to produce the data race.
pitrou
requested changes
Jan 29, 2025
cpp/src/arrow/record_batch_test.cc
Outdated
| auto schema = ::arrow::schema({field("f1", utf8())}); | ||
| auto record_batch = RecordBatch::Make(schema, length, {array_data}); | ||
| std::atomic_bool start_flag{false}; | ||
| std::thread t([record_batch, &start_flag]() { |
There was a problem hiding this comment.
Yes, this should use several threads that would do the same thing concurrently.
added 3 commits
January 29, 2025 19:55
colin-r-schultz
force-pushed
the
recordbatch-columns-thread-safety
branch
from
87216a6 to
41e23d7
Compare
|
Thanks for taking the time to review my PR! Let me just clarify the intent behind my test case. It's a minimal example that produces a data race detected by TSAN. The data race only needs to occur between 2 threads, in this case I use the main thread and
Now, we simultaneously have T1 reading There isn't an assertion that will catch this case because it is impossible for Test outputRunning the test again with the proposed fix shows no data race. The Not sure if it makes sense to have a test that only works under TSAN, but I don't think there is any way to surface the bug consistently without tooling. |
github-actions
bot
added
awaiting committer review
pitrou
reviewed
Feb 4, 2025
pitrou
reviewed
Feb 4, 2025
There was a problem hiding this comment.
Thanks for the explanation @colin-r-schultz . I understand the better and I agree the fix looks fine. I have just posted a couple suggestions.
Co-authored-by: Antoine Pitrou <[email protected]>
|
@github-actions crossbow submit -g cpp |
pitrou
approved these changes
Feb 5, 2025
|
Revision: 1f8e12e Submitted crossbow builds: ursacomputing/crossbow @ actions-719bfbc8cf
|
|
@github-actions crossbow submit -g cpp |
|
Revision: ac82bd0 Submitted crossbow builds: ursacomputing/crossbow @ actions-75e9839ac4
|
|
Merged. Thanks a lot for this @colin-r-schultz ! |
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 27900a6. There were 8 benchmark results with an error:
There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
GH-45371
What changes are included in this PR?
Use
std::atomic_compare_exchangeto initializeboxed_columns_[i]so they are correctly written only once. This means that a reference toboxed_columns_is safe to read after each element has been initialized.Are these changes tested?
Yes, there is a test case
TestRecordBatch.ColumnsThreadSafetywhich passes under TSAN.Are there any user-facing changes?
No
This PR contains a "Critical Fix".
Without this fix, concurrent calls to
SimpleRecordBatch::columnscould lead to an invalid memory access and crash.SimpleRecordBatch::columns#45371