Description
Description
Remove rephrasers
module from ragbits-document-search
package and move rerankers
to the ragbits-core
.
Motivation
Currently retrieval
module contains two things - rephrasers
and rerankers
. Most of the time we can't use provided rephrasers because the api is too limitted to do an effective rephrasing and this step ends up outside the document search pipeline. Rerankers are fine, but they are tied to the output of the document search and I can't decide what to put inside. Also the fact that this api is tied to the document search is kinda misleading, because I can rerank many things, not just documents. That's why moving it to the core seems most reasonable to me.
This idea also works well with the ragbits-piplines/ragbit-flow package, where we would put all the stuff inside the ragbits DAG, and it seems to be the best way to structure this retrieval workflow.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status