Add filtered queries. #64
Conversation
These new commands allow running queries against a filter that matches 1% or 10% of documents. Filters are interesting because some optimizations that are easy/obvious for exhaustive evaluation become more complicated when a filter is applied. Yet filters are common, think of an e-commerce search filtered by category for instance. Only the Lucene 10.0 engine supports filtering for now because I'm not too familiar with Rust, but I assume that it should be easy to add support for it in a similar fashion.
|
Thanks for the PR. I think we could make the handling slightly simpler when we move the WHERE clause to and extra "filter" field next to "query". What do you think? |
|
Can you explain a bit more what you have in mind? It was convenient to avoid introducing a third column in the protocol that src/client.py uses to submit queries to engines, or in the results web page. And then I kept the queries.txt file the same for consistency. |
I was thinking the API with a third "filter" parameter would be a little bit clearer. |
|
@PSeitz I looked into your suggestion, but having different formats in the queries.txt and the results file while doing the conversion in |
This is an alternative to #63 that adds filtered queries instead of new filtering commands.
Queries support optional filtering via the following syntax: " WHERE ", so engines can split on " WHERE " and interpret the left part as a required clause that computes scores and the right part as a filter. Only the Lucene 10 engines support this for now.