Skip to content

Support analyzer-based neural sparse query #1088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 21, 2025

Conversation

zhichao-aws
Copy link
Member

Description

Implementation for #1052

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
@zhichao-aws zhichao-aws changed the title Support analyzer-based neural sparse query & build BERT tokenizer as pre-defined tokenizer Support analyzer-based neural sparse query Apr 17, 2025
@zane-neo
Copy link
Collaborator

LGTM, will approve once some nit comments resolved.

@zhichao-aws zhichao-aws requested a review from yuye-aws April 21, 2025 03:36
@yuye-aws
Copy link
Member

@zhichao-aws Must we merge this PR after: opensearch-project/ml-commons#3719?

@zhichao-aws
Copy link
Member Author

@zhichao-aws Must we merge this PR after: opensearch-project/ml-commons#3719?

We can merge this first. All UTs and ITs are built using standard analyzer. I'll create a follow-up PR to add bert-based and mbert-based IT after the ml-commons PR merged.

@zhichao-aws zhichao-aws merged commit c3aac71 into opensearch-project:main Apr 21, 2025
51 checks passed
Copy link

codecov bot commented Apr 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (39b4892) to head (23db54b).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #1088       +/-   ##
============================================
- Coverage     80.94%       0   -80.95%     
============================================
  Files           132       0      -132     
  Lines          5819       0     -5819     
  Branches        937       0      -937     
============================================
- Hits           4710       0     -4710     
+ Misses          751       0      -751     
+ Partials        358       0      -358     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants