Skip to content

[RFC] Introduce Mapping Transformer #17500

Open
@bzhangam

Description

@bzhangam

Is your feature request related to a problem? Please describe

I'm working on a propose in neural plugin to simply the neural search set up. We propose a solution to introduce a new field type and user can define the model id in it. Then we will automatically generate neural search related fields like knn_vector field to the index mapping based on the model id defined in the index mapping. Now we need to figure out how to auto generate the fields and add them to the index mapping.

Describe the solution you'd like

There are couple of APIs that can modify the index mapping or index template

When we invoke those APIs we want to inject some logic transform the mapping before we store it.

There are two possible solutions to do that:
Option 1:
We leverage the ActionFilter to modify the request which can create/modify the index mapping/index template. But ActionFilter is kind over-empowered that it can be used to modify any action and we think it's not a good idea to keep using it in a wrong way.

Pros: No need to modify core.
Cons: It's not clear that we rely on ActionFilter to mange the mapping transform.

Option 2:
We introduce a new interface in the MapperPlugin to allow plugin to implement MappingTransformer. And then in the actions modifying the index mapping we invoke the MappingTransformer implemented by plugins to transform the mapping before we store it.

Pros: Clear responsibility.
Cons: Need to modify core to introduce this new interface.

We would like to do the option 2 since it's more clear. But also want to get some feedback from the community.

Thanks.

Related component

Plugins

Describe alternatives you've considered

No response

Additional context

[RFC] Support Semantic Field Type to Simplify Neural Search Set Up HLD
[RFC] Support Semantic Field Type to Simplify Neural Search Set Up LLD

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    • Status

      🆕 New

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions