-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify the logic of script sort parallel collection #124639
base: main
Are you sure you want to change the base?
Simplify the logic of script sort parallel collection #124639
Conversation
💚 CLA has been signed |
0a58652
to
fd0e227
Compare
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
test this please |
a64a1b4
to
4aef12c
Compare
I have run all related unit tests and integration tests multiple times, including |
@KunjueYu thanks! My comment above was only to trigger running all tests against this PR, as this does not happen automatically for PRs that are opened by an external contributor. I will review this shortly and provide feedback. Thanks again. |
This pr introduced a simpler way to support parallel collection for script sort.
The concurrency issue of script sort parallel collection was fixed in Fix concurrency issue in ScriptSortBuilder, which introduced a concurrentMap to save the relationship between leafContext and leafScript, so that the correct leafScript can be obtained later. However, we could fix the concurrency issue in another way, which may get rid of the map and simplify the code.
Before Fix concurrency issue in ScriptSortBuilder, the leaf script was a member of anonymous subClass of
BytesRefFieldComparatorSource
orDoubleValuesComparatorSource
, so that it's shared across the shard and is not thread-safe.If we create a named subClass of
BytesRefFieldComparatorSource
orDoubleValuesComparatorSource
, we can put the declaration of leaf script member in the FieldComparator (forBytesRefFieldComparatorSource
) or LeafFieldComparator (forDoubleValuesComparatorSource
). So that different comparator could hold a different reference of leaf script.In this pr, the leaf Comparator isn't wrapped as Fix concurrency issue in ScriptSortBuilder did because in each slice, the leafs are searched sequentially. So that we could set leaf script as a member of the FieldComparator, and the leaf script will be assigned to a new values for each leaf context and consumed later in the
setScorer
method, which works the same way just as before.