Skip to content

[BUG] AgentMemory.retrieve() discards every vector-store hit when the store is a VectorStore #1724

Description

@Chaoqi31

Bug Description

AgentMemory.retrieve() queries the vector store and then throws the hits away whenever the store is a semantica.vector_store.VectorStore. Long-term memories can only be found through the keyword match on the short-term buffer or the keyword fallback, so embedding search never contributes to a result.

AgentContext.retrieve() without a knowledge graph returns AgentMemory.retrieve() directly (agent_context.py:592), and ContextRetriever reads memory through the same method (context_retriever.py:973, :1409).

Steps to Reproduce

from semantica.context import AgentMemory
from semantica.vector_store import VectorStore

vs = VectorStore(backend="inmemory")
mem = AgentMemory(vector_store=vs, short_term_limit=1)
mem.store("The reactor coolant pump tripped on Tuesday")
mem.store("Quarterly revenue grew by twelve percent")

query = "nuclear plant outage"   # shares no word with either memory
print("vector store hits:", [(h["id"], round(float(h["score"]), 3))
                             for h in vs.search_vectors(query_vector=vs.embed(query), k=10)])
print("AgentMemory.retrieve:", mem.retrieve(query, max_results=5))

Expected Behavior

retrieve() returns the memories the vector store ranked for the query (both are above the default min_score=0.0).

Actual Behavior

vector store hits: [('vec_0', 0.742), ('vec_1', 0.732)]
AgentMemory.retrieve: []

Root cause

semantica/context/agent_memory.py:498-548. The loop that turns vector hits into results is indented under the second branch:

if hasattr(self.vector_store, "search_vectors"):
    ...
    vector_results = [ResultObj(r) for r in raw_results]   # built, never read

elif hasattr(self.vector_store, "search"):
    vector_results = self.vector_store.search(query=query, limit=max_results * 2)

    for result in vector_results:                          # only runs in this branch
        memory_id = result.id
        ...
        results.append({...})

VectorStore has search_vectors, so the first branch always runs and its results are discarded. The search_vectors branch was added in da08354 in front of the older search code, and the loop stayed inside the older branch.

Dedenting the loop is not enough on its own. result.id is the vector-store id (vec_0), while the loop looks it up in self.memory_items, whose keys are memory ids (mem_...). _store_memory_vector (agent_memory.py:828-864) records the mapping the other way round, in self._vector_ids[memory_id], so the loop needs a reverse lookup.

No test under tests/context/ calls search_vectors, so the dropped branch is never checked.

Suggested fix

  • Move the result loop out of the elif so both branches feed it.
  • Map vector ids back to memory ids, either with a reverse index built from self._vector_ids or by storing memory_id in the vector metadata and reading it from the hit.
  • Add a test with a real VectorStore(backend="inmemory") and a query that shares no words with the stored text, as in the repro.

Environment

  • semantica main at b14a2b8 (same code in 0.7.0)
  • Python 3.11.15, macOS arm64, core dependencies only

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions