Skip to content

[v1.14] Add composite embedders and pooling for huggingface embedders #626

Open
@Strift

Description

@Strift

This issue follows the changes related to Meilisearch v1.14.0 mega issue.

Description

Update embedders API to be compatible with Meilisearch v1.14.0

For more context, see the related issue in Meilisearch: meilisearch/meilisearch#5343

Tasks

  • Add new pooling parameter for huggingFace embedders
  • Add new composite source for embedders

For reference, here's how this works using Meilisearch JS SDK:

client
  .index('index_name')
  .updateEmbedders({
    embedder_name: {
      source: "composite",
      searchEmbedder: {
        source: "huggingFace",
        model:
          "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
        pooling: "useModel",
      },
      indexingEmbedder: {
        source: "huggingFace",
        model:
          "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
        documentTemplate: "{{doc.title}}",
        pooling: "useModel",
        documentTemplateMaxBytes: 500,
      },
    },
  });

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions