feat: add Lumina vector index read infrastructure#267
feat: add Lumina vector index read infrastructure#267XiaoHongbo-Hope wants to merge 2 commits intoapache:mainfrom
Conversation
XiaoHongbo-Hope
changed the title
XiaoHongbo-Hope
changed the title
XiaoHongbo-Hope
changed the title
XiaoHongbo-Hope
force-pushed
the
support_lumina
branch
3 times, most recently
from
a7b9e8b to
2c6608c
Compare
XiaoHongbo-Hope
force-pushed
the
support_lumina
branch
from
2c6608c to
a002256
Compare
| } | ||
|
|
||
| Ok(Some(Box::new(DictBasedScoredIndexResult::new( | ||
| id_to_scores, |
There was a problem hiding this comment.
Here can just return ids like tantivy. We don't need to introduce DictBasedScoredIndexResult.
|
|
||
| pub type ScoreGetter = Arc<dyn Fn(u64) -> f32 + Send + Sync>; | ||
|
|
||
| pub trait GlobalIndexResult: Send + Sync { |
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
You can just implement whole feature up to datafusion integration in this PR.
| storage-oss = ["opendal/services-oss"] | ||
| storage-s3 = ["opendal/services-s3"] | ||
| storage-hdfs = ["opendal/services-hdfs-native"] | ||
| lumina = ["libloading"] |
There was a problem hiding this comment.
libloading is already an indirect dependency of the project (brought in via other crates). If you add it as a direct dependency, it won't introduce any new crates to the build—it simply adds an explicit entry to your Cargo.toml. The impact on the package size is essentially negligible.
So, no need to add lumina feature.
luoyuxia
left a comment
luoyuxia
left a comment
There was a problem hiding this comment.
@XiaoHongbo-Hope Thanks for the pr. I left minor comments. PTAL
|
|
||
| let mut searcher_options = strip_lumina_options(&self.options); | ||
| for (k, v) in index_meta.options().iter() { | ||
| searcher_options.insert(k.to_string(), v.to_string()); |
There was a problem hiding this comment.
is it expected that searcher_options will overwrite self.options which i assume is the option passed by users.
3 participants
Purpose
Subtask of #227
Brief change log
Tests
API and Format
Documentation