SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

epugh · 2023-10-10T20:08:34Z

https://issues.apache.org/jira/browse/SOLR-17023

Description

Code and BATS test that demonstrates downloading a sentiment classification model from Huggingface, converting it to Onnx model, uploading the model to Solr's FileStore, and then configuring a collection and associated pipeline to use the model for enriching content.

Solution

New DocumentCategorizerUpdateProcessorFactory that interacts with the model.

Tests

BATS test. Check this PR out and run: ./gradlew integrationTests --tests test_opennlp.bats

Checklist

Please review the following and check all that apply:

[x ] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
I have created a Jira issue and added the issue ID to my pull request title.
[x ] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[ x] I have developed this patch against the main branch.
I have run ./gradlew check.
I have added tests for my changes.
I have added documentation for the Reference Guide
Look at GPU support
Integrate with Films demo.
Can we have BATS tests that depend on external resources, or do I check in the ONNX model instead? (make a stupid one???)
Write a unit test?
Provide both a label but ALSO a score!

epugh · 2023-10-10T20:10:03Z

Check it out @jzonthemtn

risdenk · 2023-10-10T20:16:44Z

solr/modules/analysis-extras/build.gradle

+configurations.all {
+  resolutionStrategy {
+    force 'org.apache.opennlp:opennlp-tools:2.2.0'
+    force 'org.apache.opennlp:opennlp-dl:2.2.0'
+  }
+}
+


Why do you need for force a resolution? This shouldn't be necessary at all...

We need apache/lucene#448 to be merged to get us to 2.2, or we ahve to work around Lucene... Is there a better way? This is what Jeff and I hacked in..

try org.apache.opennlp:opennlp*=2.2.0 in versions.props and remove this from here. It should force the upgrade to 2.2.0.

lots of errors when I took out the resolutionStrategy:

* What went wrong: Could not determine the dependencies of task ':solr:modules:analysis-extras:compileJava'. > Could not resolve all task dependencies for configuration ':solr:modules:analysis-extras:compileClasspath'. > Could not resolve org.apache.opennlp:opennlp-tools:2.2.0. Required by: project :solr:modules:analysis-extras project :solr:modules:analysis-extras > org.apache.opennlp:opennlp-dl:2.2.0 > Cannot find a version of 'org.apache.opennlp:opennlp-tools' that satisfies the version constraints: Dependency path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.opennlp:opennlp-tools:2.2.0' Constraint path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.solr:core:10.0.0-SNAPSHOT' (apiElements) --> 'org.apache.opennlp:opennlp-tools:1.9.4' because of the following reason: Computed from com.palantir.consistent-versions' versions.lock in solr-root Constraint path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.solr:solrj:10.0.0-SNAPSHOT' (apiElements) --> 'org.apache.opennlp:opennlp-tools:1.9.4' because of the following reason: Computed from com.palantir.consistent-versions' versions.lock in solr-root Dependency path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.lucene:lucene-analysis-opennlp:9.8.0' (compile) --> 'org.apache.opennlp:opennlp-tools:1.9.1'

also, feel free to push up changes to this branch ;-)

make sure to run ./gradlew --write-locks to make sure it takes into account versions.props changes.

risdenk · 2023-10-10T20:30:08Z

versions.props

@@ -70,3 +70,4 @@ org.semver4j:semver4j=5.2.1
 org.slf4j:*=2.0.9
 org.xerial.snappy:snappy-java=1.1.10.5
 software.amazon.awssdk:*=2.20.155
+org.apache.opennlp:opennlp-dl=2.2.0


try org.apache.opennlp:opennlp*=2.2.0

…t 600 mb file.

epugh · 2023-10-10T21:13:37Z

Thanks @risdenk , your suggestions worked. One slight wrinkle that I don't know how to deal with is that the com.microsoft.onnxruntime:onnxruntime_gpu:1.14.0 file doesn't work on OSX, only linux and windows. So if you look at the BATS test, you'll see some code where I replace it with com.microsoft.onnxruntime:onnxruntime:1.14.0. I don't really know how to handle that going forward.. I mean, swapping a magic jar is probably a bad idea ;-).

epugh · 2023-10-10T21:13:56Z

./gradlew integrationTests --tests test_opennlp.bats

epugh · 2023-10-10T22:46:36Z

> Task :solr:packaging:integrationTests
Running BATS tests with Solr base port 53065
1..1
ok 1 Check lifecycle of sentiment classification in 20000ms

This is after uploading the model to the PackageStore and reference it in there!

versions.props

risdenk · 2023-10-11T00:44:50Z

solr/packaging/test/test_opennlp.bats

+@test "Check lifecycle of sentiment classification" {
+
+  # GPU versions is linux and windows only, not OSX.  So swap jars.
+  rm -f ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime_gpu-1.14.0.jar


Why not just always run on cpu? Why need to test if there is a capable GPU or not? Could be on a VM without a GPU too even w/ linux and windows...

for the purposes of this, I totally agree.. I think I need some Gradle help to figure this out ;-( I've banged at it with no luck.

Just CPU is definitely a better choice here. And just a note, ONNX Runtime does not currently support Mac silicon so this will only work on x64 machines.

OpenNLP brings in the onnxruntime-gpu dependency. I will open a PR in OpenNLP to use onnxruntime instead for all of the stated reasons.

if there is a workaround, please let me know...!

Release is done. New artifacts should be available soon ;-) (not depending on gpu anymore)

Thank you!!!

risdenk · 2023-10-11T00:44:58Z

solr/packaging/test/test_opennlp.bats

+  rm -f ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime_gpu-1.14.0.jar
+  # restore 
+  #curl --insecure -o ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime-1.14.0.jar https://repo1.maven.org/maven2/com/microsoft/onnxruntime/onnxruntime/1.14.0/onnxruntime-1.14.0.jar
+  cp /Users/epugh/Documents/projects/solr-epugh/onnxruntime-1.14.0.jar ${SOLR_TIP}/modules/analysis-extras/lib/


This is using your path...

yeah... I was in the back of David Smiley's car in the wilds of New Brunswick Canada while I was working on this and didn't want to burn up my data plan ;-). Likewise I need to figure out a better apporoach with the model since it's 600 mb.... I don't know if I can find a toy one that is super small, or if I add docs to download it from hugging facet etc. Longer term, I doin't even think this bats test gets committed the way it is because of all the external dependencies.. Though.. if I had a super small model that I could check into Git, maybe that owuld be okay? Like we have a binary package for some of the package tests..

epugh · 2023-10-11T15:29:06Z

@risdenk @jzonthemtn so, any ideas how to force the onnxruntime instead of the onnxruntime_gpu? I regenerated the licenses, and noticed that it created a "onnxruntime_gpu-1.14.0.jar.sha1".

epugh · 2023-10-11T15:29:55Z

also, i am seeing two errors on looking up licenses...

Execution failed for task ':solr:modules:analysis-extras:validateJarLicenses'.
> Certain license/ notice files are missing:
    - License file missing ('com.microsoft.onnxruntime:onnxruntime_gpu:1.14.0'), expected it at: /Users/epugh/Documents/projects/solr-epugh/solr/licenses/onnxruntime_gpu-LICENSE-[type].txt, where [type] can be any of [ASL, BSD, BSD_LIKE, CDDL, CPL, EPL, MIT, MPL, PD, SUN, COMPOUND].
    - License file missing ('org.apache.opennlp:opennlp-dl:2.2.0'), expected it at: /Users/epugh/Documents/projects/solr-epugh/solr/licenses/opennlp-dl-LICENSE-[type].txt or /Users/epugh/Documents/projects/solr-epugh/solr/licenses/opennlp-LICENSE-[type].txt, where [type] can be any of [ASL, BSD, BSD_LIKE, CDDL, CPL, EPL, MIT, MPL, PD, SUN, COMPOUND].

epugh · 2023-10-11T16:05:55Z

@jzonthemtn did open https://issues.apache.org/jira/browse/OPENNLP-1515, but that will require a release of OpenNLP before we can piggy back on it

epugh · 2023-10-11T16:21:28Z

I think the first draft is done, and it "works". There are some caveats and things to be figured out, like the onnxruntime_gpu versus onnxruntime issue. Based on my work so far, we definitly need tutorials, there are too many moving pieces. The PackageStore is great, at least as far as I've worked with it, but it does expect to load the data in memory as it moves it around, so we need to figure that out. Right now we only get sentiment labels back, and we really need sentiment scores, i.e a number. I don't know if that can be a 0 to 1, of if it has to be 1,2,3,4,5 labels. I'd like to experiment with the saem thing as a copyField into a text analyzer, so you copy "review" into "review_sentiment_score" and get a number there. Or, do we have a streaming expression you run that updates the review_sentiment_score for all docs that don't have it with an atomic update?

…n the OpenNLP project

github-actions · 2024-04-17T00:00:43Z

This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the [email protected] mailing list. Thank you for your contribution!

github-actions · 2024-10-07T00:00:54Z

This PR is now closed due to 60 days of inactivity after being marked as stale. Re-opening this PR is still possible, in which case it will be marked as active again.

epugh · 2024-10-07T10:15:00Z

this is still valid ;-)

epugh · 2025-02-23T13:54:24Z

I updated this PR with main, to see what happened, and we're closer. Things that I think are still holding us back:

Solr Main is NOT on Lucene 10, which means we have to override the version of OpenNLP that Lucene uses (maybe somehow?). However when SOLR-17631: Upgrade main to Lucene 10.x #3053 gets in that should deal with it.
Need to write a JUnit test for DocumentCategorizerUpdateProcessorFactory (oops!)

epugh · 2025-02-24T15:58:03Z

...ras/src/java/org/apache/solr/update/processor/DocumentCategorizerUpdateProcessorFactory.java

+        // Initialize the categorizer.
+        FileStore fs = req.getCoreContainer().getFileStore();
+
+        var path = solrHome.resolve(ClusterFileStore.FILESTORE_DIRECTORY);


@gerlowskija this is the line I was referenceing, where the below modelFile and vocabFile feel like they are not using the ClusterFileStore properly....

epugh · 2025-02-24T20:28:42Z

Yep, need to wait for Lucene 10, otherwise we get some unit test failures:

gradlew test --tests TestOpenNLPExtractNamedEntitiesUpdateProcessorFactory.testExtractFieldRegexReplaceAll -Dtests.seed=F5DD0B40AC590A66 -Dtests.locale=pt-GW -Dtests.timezone=PRT -Dtests.asserts=true -Dtests.file.encoding=UTF-8

>     java.lang.NoSuchMethodError: 'opennlp.tools.util.Span[] opennlp.tools.sentdetect.SentenceDetectorME.sentPosDetect(java.lang.String)'
   >         at __randomizedtesting.SeedInfo.seed([F5DD0B40AC590A66:4351D7F53AC9F7A4]:0)
   >         at org.apache.lucene.analysis.opennlp.tools.NLPSentenceDetectorOp.splitSentences(NLPSentenceDetectorOp.java:41)
   >         at org.apache.lucene.analysis.opennlp.OpenNLPSentenceBreakIterator.setText(OpenNLPSentenceBreakIterator.java:199)
   >         at org.apache.lucene.analysis.util.SegmentingTokenizerBase.reset(SegmentingTokenizerBase.java:89)

epugh added 3 commits October 10, 2023 12:31

first version w bats test

f19f660

not positive I need this

48fd37b

new name, and OpenNLP is kind of a implmentation detail ;-).

cecd4f6

epugh marked this pull request as draft October 10, 2023 20:08

risdenk requested changes Oct 10, 2023

View reviewed changes

risdenk reviewed Oct 10, 2023

View reviewed changes

epugh added 2 commits October 10, 2023 18:04

baby steps, found that packagesstore blows up killing solr when i pos…

10abbd0

…t 600 mb file.

remove resolutionStraregy force from gradle build..

dd4cc89

epugh requested a review from risdenk October 10, 2023 21:16

match name in the underlying OpenNLP project. bikeshedding!

af18d0c

tidy

6950629

risdenk requested changes Oct 11, 2023

View reviewed changes

epugh added 3 commits October 11, 2023 12:22

reorder params

1adba22

log formatting

bcbf16d

regenerate...

e634b1c

dynamically grab the models from hugging face.

778e3f4

use logging structure for stack traces

405153b

cpoerschke mentioned this pull request Oct 11, 2023

[experimental] start BERT module #1073

Draft

jzonthemtn mentioned this pull request Oct 12, 2023

OPENNLP-1515: Changing onnx dependency to onnxruntime. apache/opennlp#551

Merged

10 tasks

download the correct jar, and document the work to remove this need i…

4e54090

…n the OpenNLP project

epugh changed the title ~~SOLR-17023: Use Modern NLP Models from Apache OpenNLP with Solr~~ SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr Oct 26, 2023

epugh mentioned this pull request Oct 30, 2023

SOLR-16835: Generate Python client from OpenAPI spec #1681

Merged

7 tasks

undo 'tentative: minimum Java17 for this PR' -- see PR 1510 instead

476a7d4

github-actions bot added the stale PR not updated in 60 days label Apr 17, 2024

github-actions bot added the closed-stale Closed after being stale for 60 days label Oct 7, 2024

github-actions bot closed this Oct 7, 2024

epugh reopened this Oct 7, 2024

github-actions bot added the cat:index label Oct 7, 2024

epugh added exempt-stale Prevent a PR from going stale and removed stale PR not updated in 60 days cat:index closed-stale Closed after being stale for 60 days labels Oct 7, 2024

Merge remote-tracking branch 'upstream/main' into SOLR-17023

86dd0aa

Migrate to new way of specifying dependency versions

8bffd15

github-actions bot added documentation Improvements or additions to documentation module:extraction module:sql cat:index module:langid labels Feb 23, 2025

epugh added 4 commits February 23, 2025 09:36

New home for FILESTORE_DIRECTORY constant

f374b4a

lint

36fbb39

Add Onnx to list of dependencies

c623363

Update test for recent Solr changes.

354653d

epugh commented Feb 24, 2025

View reviewed changes

Satisfy linking warning

e61296f

epugh requested review from risdenk and cpoerschke February 24, 2025 20:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

epugh commented Oct 10, 2023 •

edited

Loading

epugh commented Oct 10, 2023

risdenk Oct 10, 2023

epugh Oct 10, 2023

risdenk Oct 10, 2023

epugh Oct 10, 2023

risdenk Oct 10, 2023

risdenk Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

risdenk Oct 11, 2023

epugh Oct 11, 2023

jzonthemtn Oct 11, 2023 •

edited

Loading

epugh Oct 11, 2023

rzo1 Nov 27, 2023

epugh Nov 27, 2023

risdenk Oct 11, 2023

epugh Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

github-actions bot commented Apr 17, 2024

github-actions bot commented Oct 7, 2024

epugh commented Oct 7, 2024

epugh commented Feb 23, 2025

epugh Feb 24, 2025

epugh commented Feb 24, 2025

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

Are you sure you want to change the base?

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

Conversation

epugh commented Oct 10, 2023 • edited Loading

Description

Solution

Tests

Checklist

epugh commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jzonthemtn Oct 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

github-actions bot commented Apr 17, 2024

github-actions bot commented Oct 7, 2024

epugh commented Oct 7, 2024

epugh commented Feb 23, 2025

Choose a reason for hiding this comment

epugh commented Feb 24, 2025

epugh commented Oct 10, 2023 •

edited

Loading

jzonthemtn Oct 11, 2023 •

edited

Loading