Skip to content

Latest commit

 

History

History
322 lines (242 loc) · 5.74 KB

NMSLibSearch.md

File metadata and controls

322 lines (242 loc) · 5.74 KB

TFSimilarity.indexer.NMSLibSearch

Efficiently find nearest embeddings by indexing known embeddings and make

Inherits From: Search, ABC

TFSimilarity.indexer.NMSLibSearch(
    dims: int,
    algorithm: str = nmslib_hnsw,
    **kwargs
)

them searchable using the Approximate Nearest Neigboors Search search library NMSLIB.

Args

distance the distance used to compute the distance between embeddings.
dim the size of the embeddings.
verbose be verbose.

Methods

add

View source

add(
    embedding: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    idx: int,
    verbose: int = 1,
    build: bool = True,
    **kwargs
)

Add an embedding to the index

Args
embedding The embedding to index as computed by the similarity model.
idx Embedding id as in the index table. Returned with the embedding to allow to lookup the data associated with a given embedding.
verbose Be verbose. Defaults to 1.
build Rebuild the index after the addition. Required to make the embedding searchable. Set to false to save time between successive addition. Defaults to True.

batch_add

View source

batch_add(
    embeddings: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    idxs: Sequence[int],
    verbose: int = 1,
    build: bool = True,
    **kwargs
)

Add a batch of embeddings to the search index.

Args
embeddings List of embeddings to add to the index.

idxs (int): Embedding ids as in the index table. Returned with the embeddings to allow to lookup the data associated with the returned embeddings.
verbose Be verbose. Defaults to 1.
build Rebuild the index after the addition. Required to make the embeddings searchable. Set to false to save time between successive addition. Defaults to True.

batch_lookup

View source

batch_lookup(
    embeddings: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    k: int = 5
) -> Tuple[List[List[int]], List[List[float]]]

Find embeddings K nearest neighboors embeddings.

Args
embedding Batch of query embeddings as predicted by the model.
k Number of nearest neighboors embedding to lookup. Defaults to 5.

load

View source

load(
    path: str
)

load index on disk

Args
path where to store the data

lookup

View source

lookup(
    embedding: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    k: int = 5
) -> Tuple[List[int], List[float]]

Find embedding K nearest neighboors embeddings.

Args
embedding Query embedding as predicted by the model.
k Number of nearest neighboors embedding to lookup. Defaults to 5.

save

View source

save(
    path: str
)

Serializes the index data on disk

Args
path where to store the data