Implement hybrid search in the Conversational plugin

[KlimTodrik]

Proposal:

Based on this research, the most suitable option is to use hybrid search in the Conversational plugin. We also need to prove that this does not cause any significant degradation on the existing quality datasets.

Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

Details

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed
• OpenAPI YAML updated and issue created to rebuild clients

[KlimTodrik]

In this research, I compared three approaches:

1. Direct Hybrid Search — the original query is sent to KNN, while LLM-generated tags are used as an additional retrieval signal.
2. Original Conversational Search — the current implementation, where the LLM rewrites the query before sending it to KNN.
3. Improved Conversational Search — combines hybrid search with an additional LLM decision step that determines whether query refinement is necessary before KNN execution.

The results are promising, especially for eCommerce datasets (which is important because ESCI is not a conversational dataset):

Method	MRR@10	Precision@10
Direct Hybrid Search	0.5673	0.3033
Original Conversational Search	0.2783	0.1033
Improved Conversational Search	0.4061	0.1267

However, the same approach leads to a significant quality degradation on the FIQA conversational dataset, where we can evaluate conversational search quality directly.

Method	MRR@10
Original Conversational Search	0.4298
Improved Conversational Search	0.3276

Based on these results, I would not recommend enabling hybrid search as the default behavior for conversational search at this stage. While it significantly improves retrieval quality on eCommerce datasets, it negatively affects conversational search quality. Until we can reliably measure and balance both search quality and conversational quality, adopting this approach globally would be risky.

That said, we could expose this functionality as an optional parameter in CALL CHAT, allowing developers to choose the search strategy that best fits their use case. This would put the decision in the hands of application developers while preserving the current behavior as the default. At the moment, such a switch does not exist and remains only a potential future enhancement.

[github-actions]

Thanks for the update! However, the internal checklist is not yet complete. A Manticore team member will close this issue once everything is checked off. 🛠️