fix(pgvector): convert cosine distance to similarity score#4994
Open
Kblack0610 wants to merge 1 commit intomem0ai:mainfrom
Open
fix(pgvector): convert cosine distance to similarity score#4994Kblack0610 wants to merge 1 commit intomem0ai:mainfrom
Kblack0610 wants to merge 1 commit intomem0ai:mainfrom
Conversation
`PGVector.search()` was returning the raw `vector <=> %s::vector` distance as `OutputData.score`, but `score_and_rank()` in the hybrid retrieval pipeline (mem0/utils/scoring.py) treats `score` as a similarity (higher = better) and sorts `reverse=True`. The result is that semantically closest memories rank LAST in `Memory.search()` output, while least-similar memories surface first — easily reproduced with any small set of memories where one matches the query exactly. Convert distance → similarity at the boundary (`1.0 - r[1]`) so the score returned matches the convention every other vector store backend in this repo already uses (e.g., Chroma's distances field is also documented as such in OutputData but ranking-time inversion happens elsewhere; pgvector was the outlier that fed raw distance into the ranker). Mirrors the equivalent TypeScript SDK fix in mem0ai#4944, which described the same symptom on the JS side ("most similar documents have to be ranked highest [...] the scores were inverted, surfacing the least relevant documents"). Updated the four mocked-cursor assertions in `tests/vector_stores/test_pgvector.py` to expect the converted score (mocked distance 0.1 → score 0.9, etc.) and switched to `assertAlmostEqual` since the value is now computed. Verified end-to-end in a self-hosted deployment: before the fix, an exact-text-match memory for "plan mode default for non-trivial tasks" scored 0.221 and ranked dead last out of 19 candidates; after the fix, it ranks first at 0.779 with a meaningful gap to the next result.
|
|
Kblack0610
added a commit
to Kblack0610/home-config
that referenced
this pull request
Apr 28, 2026
Opened the Python equivalent of the merged TS fix (#4944) at mem0ai/mem0#4994 — update both the README patches table and the deployment.yaml comment so the stopgap can be dropped without re-research once #4994 merges and we bump the image.
Contributor
|
Please sign the cla and remove the comments for consistency |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PGVector.search()returns the rawvector <=> %s::vectordistance asOutputData.score, butscore_and_rank()inmem0/utils/scoring.py(used byMemory.search()for hybrid retrieval) treatsscoreas a similarity (higher = better) and sortsreverse=True. The result: semantically closest memories rank LAST while least-similar memories surface first.This is the Python-side equivalent of the same bug fixed in the TypeScript SDK in #4944, which described the symptom identically:
Reproduction
Set up mem0 with the pgvector backend, add a handful of memories, then query for the exact text of one of them. Without this fix, the exact-match memory ranks dead last with a low score; the highest-scoring result is whichever memory has the largest cosine distance from the query.
In a small self-hosted deployment (19 memories), before the fix:
After the fix:
The same pattern reproduces across queries — exact-text matches for
kubectl context home cluster,shell neovim notes, andpostgres fsGroup capabilitiesall moved from bottom-of-stack to top-of-stack after the fix.Direct postgres confirmation that the embeddings themselves are fine:
So the cause is purely the score convention mismatch between
pgvector.py:251andscore_and_rank.Fix
Convert distance → similarity at the boundary in
PGVector.search():This matches the convention every other ranker call site assumes and aligns with the TS fix in #4944.
keyword_search()is unchanged — it already returnsts_rank_cdwhich is a similarity score.Type of Change
Tests
Updated the four mocked-cursor assertions in
tests/vector_stores/test_pgvector.pyto expect the converted score (mocked distance 0.1 → score 0.9, etc.) and switched toassertAlmostEqualsince the value is now computed:$ python -m pytest tests/vector_stores/test_pgvector.py ============================== 50 passed in 0.50s ==============================Linked Issue / PR