feat: add PGVectorStore — pgvector implementation of VectorStoreInter…#377
feat: add PGVectorStore — pgvector implementation of VectorStoreInter…#377theap06 wants to merge 4 commits into
Conversation
…face Adds PostgreSQL pgvector as a vector store backend for GenericRAGAdapter. Supports cosine/L2/inner-product distance, hybrid search via tsvector, JSONB metadata filtering, HNSW indexing, and upsert/delete operations. Install with: pip install "gepa[pgvector]" Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- hybrid_search: use self._vector_score_sql() instead of hardcoded <=> so l2 and inner_product distance metrics produce correct hybrid scores - vector_search: normalize all scores to [0,1] — cosine uses GREATEST(0,1-dist), l2 uses 1/(1+dist), inner_product clamps dot product to [0,1] - get_collection_info: add rollback() in except block to avoid leaving the connection in an aborted transaction state - get_collection_info: add pg_namespace join to avoid wrong dimension when multiple schemas share the same table name - add_documents: use uuid4() for auto-generated IDs to prevent silent overwrite when add_documents is called multiple times without explicit IDs - _build_where_clause: fix bool list filter — str(True)='True' but JSONB stores 'true'; convert bools to lowercase string explicitly - create_local: replace f-string DSN with psycopg2.connect() keyword args to handle passwords/usernames containing '@', '/', '?' without URL-parsing errors Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Changed Files
|
|
Reviewed the PGVectorStore implementation. Overall structure matches the sibling stores well and the SQL is parameterized correctly (good use of A few things worth fixing or discussing before merge: Blocking
Correctness / behavior
Design / cleanup
Smaller nits
Items 1 and 2 are the ones I'd want fixed before merge. Items 3–5 are real bugs but ones you could fix in a follow-up if you're trying to land this quickly. The rest are polish. Nice clean implementation overall — the parameterized SQL is genuinely safer than I usually see in vector-store PRs. |
- Revert uv.lock to main — CI manages the lock file - zip strict=True in add_documents — lengths are already validated above, strict=True correctly asserts that invariant instead of silently truncating - Add tsvector_language parameter (default: "english") to __init__ and all factory methods; hybrid_search now uses it via sql.Literal so non-English corpora work correctly - Add GIN index on metadata in create_table to avoid sequential scans on the JSONB filter predicates produced by _build_where_clause - get_collection_info: use dimension=None instead of 0 when the column lookup returns nothing, so callers can distinguish missing from zero Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
| row = cur.fetchone() | ||
| # None signals "table exists but dimension could not be determined" | ||
| # rather than returning a misleading 0. | ||
| dimension = row[0] if row else None |
There was a problem hiding this comment.
🟡 Collection info returns a non-integer dimension, breaking the inherited dimension-getter
The dimension field is set to None (pgvector_store.py:221) instead of the integer 0 expected by the interface, so the inherited get_embedding_dimension() method returns None instead of int.
Impact: Any caller relying on get_embedding_dimension() to return an integer (e.g. for arithmetic or size checks) will get None and fail.
Interface contract violation and comparison with all other stores
The VectorStoreInterface documents "dimension" as (int): Vector embedding dimension (0 if unknown) at vector_store_interface.py:170. The base-class get_embedding_dimension() at vector_store_interface.py:193-194 calls info.get("dimension", 0) — if the key exists with value None, .get() returns None rather than the default 0.
Every other store implementation (Chroma, Qdrant, Milvus, LanceDB, Weaviate) returns "dimension": 0 when the dimension is unknown. PGVectorStore returns None in both the success path (line 221) and the error path (line 235).
| dimension = row[0] if row else None | |
| dimension = row[0] if row else 0 |
Was this helpful? React with 👍 or 👎 to provide feedback.
##Summary
Adds PostgreSQL pgvector as a vector store backend for GenericRAGAdapter. Supports cosine/L2/inner-product distance, hybrid search via tsvector, JSONB metadata filtering, HNSW indexing, and upsert/delete operations.