🚀 Otto.de high performance Solr

This Solr fork repository contains bugfixes and optimizations targeted for upstream Solr. It is our working repository to file pull requests against the upstream repo. It furthermore contains bugfixes and improvements that not yet made it to the upstream repository.

📦 Improvements and bugfixes

These are the improvements and bugfixes in this release. Some are already filed as Solr issues, some are pending issues and others have not yet been submitted.

Note

✅ checked issues have been successfully merged to upstream
⏳ waiting for approval
✨ new fix, needs to filed and pull requested

• ⏳ SOLR-10059 In SolrCloud, every fq added via <lst name="appends"> is computed twice. This breaks the collapse filter if configured. Our fix turns this in a different direction, and sanitizes macros in appended fq parameters
• ✨ SOLR-17334 Minor bugs in Solr dedicated coordinator mode. Fix access to the root resource and allow coordinator requests outside of the /select handler

Pending fixes

• ⏳ SOLR-16497 Allow finer grained locking in SolrCores

Upcoming fixes

• ✨ SOLR-xxxx Add an instrumented HttpShardHandlerFactory

Merged fixes

• ✅ SOLR-17686 Our approach to Re-enable custom stages
• ✅ SOLR-17187 Add the ability to supply a custom poll interval in the updateHandler. This is of interest for TLOG/PULL replica setups with longer commit intervals.
• ✅ SOLR-17337 Show proper distributed stage id
• ✅ SOLR-17185 Open up ValueAugmenterFactory.ValueAugmenter for extension
• ✅ SOLR-15377 Do not swallow exceptions thrown in replication
• ✅ SOLR-16489 CaffeineCache puts thread into infinite loop
• ✅ SOLR-16515 Remove synchronized access to cachedOrdMaps in SlowCompositeReaderWrapper

👩‍💻 Using this fork repository

To incorporate the bugfixed sources and Java releases in your project, add the GitHub Maven Package Repository to your Maven or Gradle file.

<project>
  <repositories>
    <repository>
      <id>otto-de-solr</id>
      <name>Otto.de Solr Fork</name>
      <url>https://maven.pkg.github.com/otto-de/solr</url>
    </repository>
  </repositories>
</project>

Docker images are published for both arm64 and amd64 architectures:

docker run -itp 8983:8983 ghcr.io/otto-de/solr:9.10.1

Note

There is no latest tag available for the Docker images

👩‍💻 Working with this fork

Our goal is to get all improvements merged into upstream. We'll file all our pull requests based on this repository. Before doing any work, please familiarize yourself with the Solr CONTRIBUTING guidelines.

✨ Adding / working on a new improvement or fix

Note

Do not pollute .gitignore settings with your local specialities. Use .git/info/exclude for local git ignores

1. If not yet present, open an issue in the Solr Jira Bugtracker. Use the issue number to label your branch and future commits.
2. Create feature/* branch for your feature that forks off the main branch
3. Add the branch-test.yaml Github Action Workflow to your new branch
4. After finishing improving Solr, run ./gradlew tidy to properly format your source code
5. Check all formalities via ./gradlew check
6. Push your new branch

🚀 Releasing a new version

Clone the fork repository and prepare the upstream source git repository:

git remote -v
git remote add upstream https://github.com/apache/solr.git
git fetch upstream

Solr 9 is a JDK11 project. Please make your adjustment using a recent JDK11:

sdk use java 11.0.30-tem

🔁 Releasing a new patch level version

If you want to release a new bugfix version of a already existing Solr release (say 9.6.0-otto-de.2 over 9.6.0-otto-de.1), follow these steps.

1. Locate the current release candidate branch. For Solr 9.6.x this would be candidates/branch_9_6
2. Sync the Apache Solr release branch branch_9_6 using the GitHub UI
3. Rebase our release candidate branch onto the Apache Solr release branch and replay all Cherry Pick Commits. Force push our release candidate branch to GitHub

git checkout candidates/branch_9_6
git rebase branch_9_6 --reapply-cherry-picks
git push origin candidates/branch_9_6 --force

4. Now cherry pick the new features/issues onto the candidate branch
5. Run ./gradlew check
6. Push changes to candidate branch

🎯 Releasing a new minor version

If you want to release a new bugfix version of a new Solr release (say 9.8.0-otto-de.1 over 9.5.0-otto-de.5), follow these steps.

1. Fork the Solr minor version release branch, e.g. branch_9_10 into our fork repository

$ git fetch upstream
$ git checkout branch_9_10
$ git push origin branch_9_10

2. Create a bugfix branch candiates/branch_9_10 branching off the Solr minor release branch

$ git checkout -b candidates/branch_9_10

3. Add our test and release Github Action Workflows to your new branch. Run the branch-test.yaml GitHub Action manually after push to check the branches baseline.

$ curl -fsLo .github/workflows/branch-test.yaml \
   "https://raw.githubusercontent.com/otto-de/solr/about-this-fork/.github/workflows/branch-test.yaml"
$ curl -fsLo .github/workflows/release.yaml \
   "https://raw.githubusercontent.com/otto-de/solr/about-this-fork/.github/workflows/release.yaml"
$ git add .github/workflows/branch-test.yaml
$ git add .github/workflows/release.yaml
$ git commit -m "Add branch test and release action"
$ git push origin candidates/branch_9_10

4. Cherry pick all fixes from the features/** branches to our candidate branch. For each fix, verify that they are still needed! Some may have been merged in the meantime! After each cherry-pick make sure things integrate well (via ./gradlew check).

# [SOLR-10059]
$ git cherry-pick a82b500d3c633621b0062698f540c2974a920fbb && ./gradlew clean compileJava compileTestJava

# instrumented shard handler
$ git cherry-pick 5091131251e8995c16606d2a6555290fc3b2735c && ./gradlew clean compileJava compileTestJava

# [SOLR-17334] Allow coordinator requests outside of /select
$ git cherry-pick cdd71a06c3e2109bf8151217b7ea036efe18655b && ./gradlew clean compileJava compileTestJava

# [SOLR-16497] Finer grained locking
$ git cherry-pick 9a514ee7fea55c769d410a9aa5424d9745cea580 && ./gradlew clean compileJava compileTestJava

# done
$ git push origin candidates/branch_9_10

Every push triggers the Branch Test GitHub Action for your new release candidate branch. As soon as the action is green, ...

5. trigger the Release GitHub Action to build and publish a new release