Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17631: Upgrade main to Lucene 10.x #3053

Open
wants to merge 36 commits into
base: main
Choose a base branch
from

Conversation

chatman
Copy link
Contributor

@chatman chatman commented Jan 22, 2025

SOLR-17631

Upgrade main to Lucene 10.x

Right now, I've fixed all compilation issues with core and test modules. Other modules still don't compile.
On the tests front, I've fixed a few tests. There are the tests that still fail:

./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.update.processor.PreAnalyzedUpdateProcessorTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestRandomDVFaceting.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.DistributedIntervalFacetingTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.analysis.TestReversedWildcardFilterFactory.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestGroupingSearch.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestJoin.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.analysis.PathHierarchyTokenizerFactoryTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestDistributedSearch.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.BasicFunctionalityTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestDistributedGrouping.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.TestRandomFaceting.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.BasicDistributedZkTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.BasicDistributedZk2Test.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.DocValuesNotIndexedTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.TestSnapshotCoreBackup.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedSpellCheckComponentTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.TestSegmentSorting.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DebugComponentTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedFacetPivotLargeTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedFacetPivotWhiteBoxTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestTrieFacet.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.core.TestCodecSupport.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.facet.TestCloudJSONFacetSKG.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.core.SOLR749Test.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.designer.TestSchemaDesignerAPI.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.spelling.SpellCheckCollatorTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.uninverting.TestFieldCacheVsDocValues.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.update.SolrIndexConfigTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedFacetPivotSmallTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.facet.TestCloudJSONFacetJoinDomain.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedFacetPivotSmallAdvancedTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.request.TestIntervalFaceting.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.uninverting.TestFieldCacheSortRandom.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.schema.PreAnalyzedFieldTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.QueryElevationComponentTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.StatsComponentTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestComplexPhraseLeadingWildcard.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestMissingGroups.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestRealTimeGet.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.request.TestWriterPerf.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.facet.TestJsonFacetRefinement.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.request.SimpleFacetsTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.response.TestRetrieveFieldsOptimizer.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.schema.DocValuesMultiTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.MoreLikeThisHandlerTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.LargeFieldTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.schema.TestUseDocValuesAsStored.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.FacetPivotSmallTest.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.request.TestFaceting.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestDocValuesIteratorCache.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestQueryLimits.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestGraphTermsQParserPlugin.txt
./solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestFiltering.txt



Comment on lines +148 to +149
assertEquals("1", searcher.storedFields().document(hits.scoreDocs[0].doc).get("id"));
assertEquals("0", searcher.storedFields().document(hits.scoreDocs[1].doc).get("id"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere outside of modules/ltr similar changes are needed too, #3149 proposes to separately replace.

…ne-10

Resolved Conflicts:
	solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
	solr/core/src/test/org/apache/solr/core/TestCodecSupport.java
@@ -603,7 +566,7 @@ protected Integer getNumberOfGroups() {
protected void populateScoresIfNecessary() throws IOException {
if (needScores) {
for (GroupDocs<?> groups : result.groups) {
TopFieldCollector.populateScores(groups.scoreDocs, searcher, query);
TopFieldCollector.populateScores(groups.scoreDocs(), searcher, query);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leaving a note here from investigating the grouping-related test failures, the filling in of the scores here does not influence the maxScore of a group and that seems to be wrong (always 1 possibly?) and so the groups are ordered wrongly.

@janhoy
Copy link
Contributor

janhoy commented Feb 22, 2025

@chatman review feedback waiting :)

@epugh
Copy link
Contributor

epugh commented Feb 23, 2025

Looking forward to this landing. I was just looking at upgrading OpenNLP and seeing that it requires Lucene 10...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment