Add new notebook for hugging face integration

[maxhniebergall]

Ready for final review

[gitnotebooks]

Found 1 changed notebook. Review the changes at https://gitnotebooks.com/elastic/elasticsearch-labs/pull/301

[joemcelroy]

Commented on notebook notebooks/integrations/hugging-face/huggingface-integration-millions-of-documents-with-cohere-reranking.ipynb Cell 24 Line 9

    index="hf-semantic-text-index",
    mappings={
        "properties": {
            "infer_field": {
                "type": "semantic_text",
                "inference_id": "my_hf_endpoint_object",
            },
            "text_field": {"type": "text", "copy_to": "infer_field"},

curious - why have the additional text_field that used to copy?

[maxhniebergall]

Thats required for semantic text. The strings go into the text_field, and the embeddings get stored in the infer_field

[joemcelroy]

you can just send it directly to infer_field instead. semantic_text doesn't require a copy_to field.

[maxhniebergall]

hmm, I guess so. But if we need access to the strings and the embeddings, don't we need the text_field? This copy_to field is used all over our semantic_text notebooks from what I've seen

[joemcelroy]

you access that all through semantic field. Behind scenes, semantic field stores the full text into a keyword field and does the chunking into another data structure.

The only time you would need two fields (text and semantic) is if you were doing hybrid search, as the text is stored in a keyword field

[maxhniebergall]

huh, we should probably update our notebooks then.

[joemcelroy]

its an interesting take because i agree, the examples with copy_to is confusing. We built semantic to operate like a normal text field and match query clause, just does semantic search instead of lexical.