SOLR-17310: Configurable LeafSorter to customize segment search order #2477
Conversation
weiwang19
changed the title
weiwang19
changed the title
solr/core/src/test-files/solr/collection1/conf/solrconfig-segmentsort.xml
Outdated
Show resolved
Hide resolved
github-actions
bot
added
the
documentation
|
Tagging @cpoerschke for review; I figure you're a better reviewer here. I haven't used this aspect of Lucene before. |
| r -> { | ||
| try { | ||
| return Long.parseLong( | ||
| ((SegmentReader) r).getSegmentInfo().info.getDiagnostics().get(TIME_FIELD)); |
There was a problem hiding this comment.
Use of the diagnostics here seems very specialised and potentially fragile. Leaf sorting is "between segment sorting" and we also have index sorting i.e. "within segment sorting" -- I wonder if there might be enough commonality to generalise. Will add more detailed scribbles on the https://issues.apache.org/jira/browse/SOLR-17310 ticket itself.
There was a problem hiding this comment.
... more detailed scribbles on the ... ticket itself.
convenience link: https://issues.apache.org/jira/browse/SOLR-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854761#comment-17854761
There was a problem hiding this comment.
Have been checking the available sources but currently the timestamp is only in the segment info diagnostics.
|
This PR has had no activity for 60 days and is now labeled as stale. Any new activity or converting it to draft will remove the stale label. To attract more reviewers, please tag people who might be familiar with the code area and/or notify the [email protected] mailing list. Thank you for your contribution! |
|
This PR has had no activity for 60 days and is now labeled as stale. Any new activity will remove the stale label. To attract more reviewers, please tag people who might be familiar with the code area and/or notify the [email protected] mailing list. To exempt this PR from being marked as stale, make it a draft PR or add the label "exempt-stale". If left unattended, this PR will be closed after another 60 days of inactivity. Thank you for your contribution! |
|
This PR is now closed due to 60 days of inactivity after being marked as stale. Re-opening this PR is still possible, in which case it will be marked as active again. |
|
keep it open |
|
Thanks for bringing this up. Would definitely like to have a discussion
about moving this forward.
Thanks,
Wei
…On Sun, Feb 2, 2025 at 5:20 AM Eric Pugh ***@***.***> wrote:
Seems like this ticket and #313 <#313>
both have stalled out... Thoughts on a thread on the dev mailing list to
see if we think moving forward with this or the #313
<#313> approach gets us moving?
—
Reply to this email directly, view it on GitHub
<#2477 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHFRMAHANN5EOQ377KR34FD2NYLQ5AVCNFSM6AAAAABIG2LSHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRZGM4TGOJYGM>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
https://issues.apache.org/jira/browse/SOLR-17310
Description
Lucene's IndexWriterConfig provides the option to sort leaf readers when a custom LeafSorter is provided. The functionality is currently not directly exposed in Solr. There are cases where we would like to customize the segment visit order, for example, visit the recently updated segments first when early termination is applied.
Solution
The SegmentTimeLeafSorter sorts the LeafReaders by time stamp, in ascending or descending order. It can be enabled by adding the segmentSort config in solrconfig.xml. Without the config, no sorting is applied by default.
Tests
Added unit test and verified all tests pass.
Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.