-
Notifications
You must be signed in to change notification settings - Fork 94
Introduce a configurable remote metadata client AND migrate LockService to the client #831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce a configurable remote metadata client AND migrate LockService to the client #831
Conversation
a6a11ae
to
a49957b
Compare
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (70.51%) is below the target coverage (75.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #831 +/- ##
==========================================
+ Coverage 70.49% 70.51% +0.02%
==========================================
Files 42 42
Lines 2057 2106 +49
Branches 165 174 +9
==========================================
+ Hits 1450 1485 +35
- Misses 495 505 +10
- Partials 112 116 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
e580318
to
a07a360
Compare
@cliu123 is it possible to spin up an external service for integration test purposes to simulate the remote store? i.e. let's run the integration tests of the sample plugin with remote store enabled. Potentially DynamoDB local? https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html Docker images located here: https://hub.docker.com/r/amazon/dynamodb-local |
157d656
to
5a4cb62
Compare
277df11
to
dc655e6
Compare
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
… failure Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
…sion catalog for consistency Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
1083c83
to
0e11b0a
Compare
Signed-off-by: cliu123 <[email protected]>
0e11b0a
to
2b935b4
Compare
Signed-off-by: cliu123 <[email protected]>
Signed-off-by: cliu123 <[email protected]>
202df9c
to
3a8754e
Compare
Signed-off-by: cliu123 <[email protected]>
3a8754e
to
ad89150
Compare
src/main/java/org/opensearch/jobscheduler/utils/LockServiceImpl.java
Outdated
Show resolved
Hide resolved
Signed-off-by: cliu123 <[email protected]>
…ockService to the client (opensearch-project#831)" This reverts commit db49a15 to avoid jarHell in downstream plugins.
Description
Currently, the Job Scheduler plugin stores job metadata in a system index on the local OpenSearch cluster. However, some deployments require more flexibility in how and where metadata is stored. There is no straightforward way to configure the plugin to store job metadata outside of the local OpenSearch cluster.
This PR does 2 things:
LockService.java
from OpenSearch transport client to SdkClient. Note that the SdkClient does not support remote index or DynamoDB table creation for now, so the index creation will be skipped when remote metadata store is configured.isMultiTenancyEnabled
is used as a proxy of enabling remote metadata store.Related Issues
Resolves #828
Check List
--signoff
.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.