Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce mapping transformer to allow transform mappings during index create/update or index template create/update #17635

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bzhangam
Copy link

Description

In this PR we introduce the mapping transformer to the core to allow transforming the mapping during index create/update and index template create/update. We want to do this because we have a use case in neural search plugin where we want to auto generate the semantic fields(e.g. knn_vector or rank_feature) in the mappings based on the model id defined in the semantic field(A new field type introduced by neural search plugin). In this way we can simplify the neural search set up.

Potentially in future this feature can be leveraged by other plugins to auto transform the mapping based on the certain config defined by users.

Related Issues

Resolves #[17500] - Proposal to introduce the mapping transformer into core.

#[803] - Proposal to support semantic field in neural search.
#[1211] - HLD of the semantic field.

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 7d47df7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bzhangam bzhangam force-pushed the introduceMappingTransformer branch 2 times, most recently from 6f9c569 to dc11eee Compare March 19, 2025 20:59
Copy link
Contributor

❌ Gradle check result for dc11eee: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bzhangam bzhangam force-pushed the introduceMappingTransformer branch from dc11eee to c35c386 Compare March 20, 2025 17:11
Copy link
Contributor

❌ Gradle check result for c35c386: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bzhangam bzhangam force-pushed the introduceMappingTransformer branch from c35c386 to 2406d5b Compare March 25, 2025 18:26
@bzhangam bzhangam requested a review from peternied as a code owner March 25, 2025 18:26
@bzhangam bzhangam closed this Mar 25, 2025
@bzhangam bzhangam force-pushed the introduceMappingTransformer branch from 2406d5b to 3ac7c23 Compare March 25, 2025 18:30
@bzhangam bzhangam reopened this Mar 25, 2025
Copy link
Contributor

❌ Gradle check result for b8204e8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bzhangam bzhangam force-pushed the introduceMappingTransformer branch from b8204e8 to 6016e32 Compare March 25, 2025 19:52
…x create/update or index template create/update.

Signed-off-by: Bo Zhang <[email protected]>
@bzhangam bzhangam force-pushed the introduceMappingTransformer branch from 6016e32 to 0103403 Compare March 25, 2025 19:55
Copy link
Contributor

❌ Gradle check result for 0103403: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cluster Manager enhancement Enhancement or improvement to existing feature or request Plugins
Projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

[RFC] Introduce Mapping Transformer
2 participants