Skip to content

Benchmark fixes #4194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dsmiley merged 9 commits into from
Mar 20, 2026
+55 −33
Merged

Benchmark fixes #4194

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
1be1998
IntegersDSL: fix initialization race
dsmiley Mar 4, 2026
2a7001d
schema.xml: _version_ needs docValues
dsmiley Mar 4, 2026
d43c568
StreamingSearch: fix leak of threads
dsmiley Mar 4, 2026
9bfb302
MiniClusterState: use ObjectReleaseTracker to help validate the bench…
dsmiley Mar 7, 2026
309be83
Fix benchmark QueryResponseWriters
dsmiley Mar 6, 2026
c7f412a
Fix QueryResponseWriters (don't use raw)
dsmiley Mar 6, 2026
e8366e4
StreamingSearch bring back httpSolrClient
dsmiley Mar 7, 2026
3370506
Fix FilterCache.dumpMetrics for prometheus
dsmiley Mar 8, 2026
c8eb02b
Merge branch 'main' into benchmark-fixes
dsmiley Mar 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions solr/benchmark/src/java/org/apache/solr/bench/MiniClusterState.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.common.util.IOUtils;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.ObjectReleaseTracker;
import org.apache.solr.common.util.SolrNamedThreadFactory;
import org.apache.solr.common.util.SuppressForbidden;
import org.apache.solr.embedded.JettySolrRunner;
Expand Down Expand Up @@ -157,6 +158,11 @@ public void shutdownMiniCluster(BenchmarkParams benchmarkParams, BaseBenchState
IOUtils.closeQuietly(client);
cluster.shutdown();
logClusterDirectorySize();

String orr = ObjectReleaseTracker.clearObjectTrackerAndCheckEmpty();
if (orr != null) {
Copy link
Contributor Author

@dsmiley dsmiley Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this helped track down 2 issues!

throw new AssertionError("ObjectReleaseTracker found unreleased objects:\n" + orr);
}
}

private void logClusterDirectorySize() throws IOException {
Expand Down
17 changes: 11 additions & 6 deletions solr/benchmark/src/java/org/apache/solr/bench/generators/IntegersDSL.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ private static class IntegerMaxCardinalitySolrGen extends SolrGen<Integer> {
private final Gen<Integer> integers;

/** The Cardinality start. */
Integer cardinalityStart;
volatile Integer cardinalityStart;

/**
* Instantiates a new Integer max cardinality solr gen.
Expand All @@ -132,13 +132,18 @@ public IntegerMaxCardinalitySolrGen(int maxCardinality, Gen<Integer> integers) {

@Override
public Integer generate(SolrRandomnessSource in) {
Copy link
Contributor Author

@dsmiley dsmiley Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be called concurrently; needs to be thread-safe

if (cardinalityStart == null) {
cardinalityStart =
SolrGenerate.range(0, Integer.MAX_VALUE - maxCardinality - 1).generate(in);
Integer localStart = cardinalityStart;
if (localStart == null) {
synchronized (this) {
localStart = cardinalityStart;
if (localStart == null) {
localStart = SolrGenerate.range(0, Integer.MAX_VALUE - maxCardinality - 1).generate(in);
cardinalityStart = localStart;
}
}
}

long seed =
SolrGenerate.range(cardinalityStart, cardinalityStart + maxCardinality - 1).generate(in);
long seed = SolrGenerate.range(localStart, localStart + maxCardinality - 1).generate(in);
return integers.generate(new SplittableRandomSource(new SplittableRandom(seed)));
}
}
Expand Down
13 changes: 11 additions & 2 deletions solr/benchmark/src/java/org/apache/solr/bench/search/QueryResponseWriters.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
import static org.apache.solr.bench.generators.SourceDSL.strings;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.solr.bench.Docs;
import org.apache.solr.bench.MiniClusterState;
import org.apache.solr.bench.MiniClusterState.MiniClusterBenchState;
Expand All @@ -30,6 +32,7 @@
import org.apache.solr.client.solrj.response.InputStreamResponseParser;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.common.util.NamedList;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
Expand Down Expand Up @@ -57,7 +60,7 @@ public class QueryResponseWriters {
@State(Scope.Benchmark)
public static class BenchState {

@Param({CommonParams.JAVABIN, CommonParams.JSON, "cbor", "smile", "xml", "raw"})
@Param({CommonParams.JAVABIN, CommonParams.JSON, "cbor", "smile", "xml"})
Copy link
Contributor Author

@dsmiley dsmiley Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"raw" is special; isn't working (NPE) in this scenario. Don't think it makes sense to benchmark it.

String wt;

private int docs = 100;
Expand Down Expand Up @@ -93,6 +96,12 @@ public void setup(MiniClusterBenchState miniClusterState) throws Exception {
public Object query(
BenchState benchState, MiniClusterState.MiniClusterBenchState miniClusterState)
throws SolrServerException, IOException {
return miniClusterState.client.request(benchState.q, collection);
NamedList<Object> response = miniClusterState.client.request(benchState.q, collection);
// consume the stream completely
Copy link
Contributor Author

@dsmiley dsmiley Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is necessary since otherwise the InputStreamResponseParser is left unconsumed and in fact it means the benchmark was mostly useless as nothing was transmitted since there was no consumer! wow

try (InputStream responseStream =
(InputStream) response.get(InputStreamResponseParser.STREAM_KEY)) {
responseStream.transferTo(OutputStream.nullOutputStream());
}
return response;
}
}
16 changes: 7 additions & 9 deletions solr/benchmark/src/java/org/apache/solr/bench/search/StreamingSearch.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import org.apache.solr.bench.Docs;
import org.apache.solr.bench.MiniClusterState;
import org.apache.solr.bench.MiniClusterState.MiniClusterBenchState;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.io.SolrClientCache;
import org.apache.solr.client.solrj.io.Tuple;
Expand All @@ -35,6 +36,7 @@
import org.apache.solr.client.solrj.jetty.HttpJettySolrClient;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.common.util.IOUtils;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
Expand Down Expand Up @@ -68,7 +70,7 @@ public static class BenchState {
private String zkHost;
private ModifiableSolrParams params;
private StreamContext streamContext;
private HttpJettySolrClient httpJettySolrClient;
private SolrClient httpSolrClient;

@Setup(Level.Trial)
public void setup(MiniClusterBenchState miniClusterState) throws Exception {
Expand Down Expand Up @@ -96,21 +98,17 @@ public void setup(MiniClusterBenchState miniClusterState) throws Exception {
@Setup(Level.Iteration)
public void setupIteration(MiniClusterState.MiniClusterBenchState miniClusterState)
throws SolrServerException, IOException {
SolrClientCache solrClientCache;
// TODO tune params?
var client = new HttpJettySolrClient.Builder().useHttp1_1(useHttp1).build();
solrClientCache = new SolrClientCache(client);

var httpSolrClient = new HttpJettySolrClient.Builder().useHttp1_1(useHttp1).build();
this.httpSolrClient = httpSolrClient;
SolrClientCache solrClientCache = new SolrClientCache(httpSolrClient);
streamContext = new StreamContext();
streamContext.setSolrClientCache(solrClientCache);
}

@TearDown(Level.Iteration)
public void teardownIt() {
streamContext.getSolrClientCache().close();
if (httpJettySolrClient != null) {
httpJettySolrClient.close();
}
IOUtils.closeQuietly(httpSolrClient);
}
}

Expand Down
2 changes: 1 addition & 1 deletion solr/benchmark/src/resources/configs/cloud-minimal/conf/schema.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
</fieldType>

<!-- for versioning -->
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="_version_" type="long" indexed="true" stored="true" docValues="true"/>
Copy link
Contributor Author

@dsmiley dsmiley Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need docValues on _version_. This ought to be a loud Solr error but it turned up here latently when a DBQ came in -- very weird.

<field name="_root_" type="string" indexed="true" stored="true" multiValued="false" required="false"/>
<field name="id" type="string" indexed="true" stored="true"/>
<field name="text" type="text" indexed="true" stored="false"/>
Expand Down
Loading