[NEW] Vector databse related support

[indranilr]

The problem/use-case that the feature addresses

Enable use of Valkey with LLM applications for semantic LLM caching, semantic conversation cache, LLM semantic routing

Description of the feature

• Introduce support for vector data types and similarity search query
• • Support following methods and engines (Method : HNSW,FLAT, Engine : NMSLIB, Faiss)
• Vector Range Search (e.g. find all vectors within a radius of a query vector)
• Support Hyrbid search (lexical and semantic search)
• Document ranking (using tf-idf, with optional user-provided weights)
• Support for JSON based representation of vectors.
• LLM Semantic Cache and Chat Session history management APIs support
• Introduce related client python library to use the vector database related function from LLM chain - integration with Langchain, haystack, llamaIndex
• Have default embedding models or use custom embedding/re-ranking, ability to integrate with HCP hosted Embedding/reranking models for the same through configuration.

Alternatives you've considered

Refer to below issue.

https://github.com/orgs/valkey-io/discussions/371

Additional information

Consider port of https://www.redisvl.com/index.html

[PingXie]

Great timing :-). There is an RFC proposal for this exact topic: valkey-io/valkey-rfc#8.

[hwware]

We have publish the repo https://github.com/valkey-io/valkey-search