Skip to content

Feat filter vectorstore #7511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

gs-costa
Copy link
Collaborator

@gs-costa gs-costa commented Apr 8, 2025

Add Filtering Capabilities to Vector Store Components

Overview

This PR adds filtering capabilities to all vector store components in Langflow, allowing users to narrow down search results based on specific field values. This enhancement improves search precision by enabling metadata-based filtering across all supported vector store backends.

Key Changes

  • Added search_filter parameter to the base LCVectorStoreComponent class
  • Implemented filter support in vector store components below:
    • Pinecone
    • Qdrant
    • Supabase
    • Milvus
    • FAISS
    • Elasticsearch
    • MongoDB (pre_filter)
    • Weaviate (where_filter)
    • Vectara
    • Upstash
  • Updated the VectorStoreRAG template to support filtering
  • Fixed filter handling in LocalDB and Elasticsearch
  • filter support not implemented due to issues:
    • Clickhouse
    • OpenSearch
    • PGVector
    • Weaviate (need client connection fix)
    • Couchbase (do not allow filter)
    • Redis

Implementation Details

  • Added NestedDictInput for search_filter in the base component
  • Updated search_with_vector_store method to pass filter parameters to vector store search methods
  • Modified each vector store component to properly handle the filter parameter
  • Ensured consistent filter parameter naming across different backends

Usage Example

# Create a filter for a specific file path
search_filter = {
    "file_path": "/path/to/document.pdf"
}

# Use in vector store component
results = redis_vector_store.search_documents(
    search_query="your query",
    search_filter=search_filter
)

Testing

  • Added tests for filter functionality across vector store components
  • Verified filter behavior with different data types and filter expressions

Impact

Enables more precise search functionality by filtering results based on metadata, improving search relevance and performance across all supported vector store backends.

@gs-costa gs-costa marked this pull request as ready for review April 8, 2025 19:44
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Apr 8, 2025
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Apr 8, 2025
@gs-costa gs-costa marked this pull request as draft April 8, 2025 21:51
@gs-costa gs-costa marked this pull request as ready for review April 10, 2025 18:40
@dosubot dosubot bot added the enhancement New feature or request label Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant