Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add semantic reranking notebook #308

Merged
merged 9 commits into from
Aug 12, 2024
Merged

Conversation

demjened
Copy link
Contributor

@demjened demjened commented Aug 7, 2024

Add notebook on semantic reranking (BM25 search + reranking with Cohere through inference endpoint).

Copy link

gitnotebooks bot commented Aug 7, 2024

Found 1 changed notebook. Review the changes at https://gitnotebooks.com/elastic/elasticsearch-labs/pull/308

@demjened demjened marked this pull request as draft August 7, 2024 21:28
},
"outputs": [],
"source": [
"url = \"https://raw.githubusercontent.com/elastic/elasticsearch-labs/main/notebooks/search/articles-wikipedia.json\"\n",
Copy link
Contributor Author

@demjened demjened Aug 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demjened demjened marked this pull request as ready for review August 9, 2024 15:11
@@ -5,6 +5,7 @@ EXEMPT_NOTEBOOKS=(
"notebooks/esql/esql-getting-started.ipynb"
"notebooks/search/07-inference.ipynb"
"notebooks/search/08-learning-to-rank.ipynb"
"notebooks/search/10-semantic-reranking.ipynb"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not testable due to dependency on Cohere API key.

Copy link
Member

@joshdevins joshdevins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor change only, otherwise LGTM

@leemthompo
Copy link
Contributor

Looks good. We already have a Cohere notebook that includes reranking instructions. Consider a sentence or two in the introduction to mention we have a full-blown Cohere notebook, and potentially mention how the retriever syntax is the differentiator in the notebook added in this PR.

@demjened
Copy link
Contributor Author

@leemthompo With regards to mentioning the other Cohere notebook, how about this text?

This example will show how to combine search and semantic reranking to improve the accuracy of your search results. We'll be using the rerank feature from Cohere.

Note: for a complete integration with Cohere please refer to this notebook. This example focuses on Cohere reranking only through an Elastic retriever query.

@leemthompo
Copy link
Contributor

Sounds good. Maybe rename the notebook to semantic-reranking-retrievers-cohere.ipynb because we'll be adding at least one more semantic reranking with retrievers notebook using different providers/Eland option.

@demjened demjened merged commit 26d07be into main Aug 12, 2024
5 checks passed
@demjened demjened deleted the demjened/semantic-reranking-notebook branch August 12, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants