Skip to content

feat(retrieve): add scoped reference-aware RAG recall - #479

Open
COMPRyanLI wants to merge 1 commit into
NevaMind-AI:mainfrom
COMPRyanLI:reference-aware-rag
Open

feat(retrieve): add scoped reference-aware RAG recall#479
COMPRyanLI wants to merge 1 commit into
NevaMind-AI:mainfrom
COMPRyanLI:reference-aware-rag

Conversation

@COMPRyanLI

Copy link
Copy Markdown

Pull Request Summary

Adds opt-in reference-aware item retrieval to RAG while preserving the existing
vector-only default and the public retrieve() response shape.


What does this PR do?

  • scans retrieved category summaries for existing [ref:...] citations
  • fetches cited items through the scoped list_items_by_ref_ids() repository lookup
  • merges referenced and vector hits while deduplicating item IDs
  • preserves user and agent scope filtering
  • respects item.top_k, including top_k=0
  • preserves vector-only behavior when reference retrieval is disabled
  • adds deterministic quality, leakage, conflict, and LangGraph integration tests

Why is this change needed?

Vector similarity can miss an exact memory item already cited by a relevant category.
Following those citations provides deterministic recall without introducing an
unconditional item scan or changing the public retrieval API.

On a deterministic three-query synthetic benchmark:

Metric Vector-only Reference-aware
Cited-item recall 0% 100%
Precision@k 0% 100%
Duplicate count 0 0
Scope leaks 0 0
Missing-reference errors 0 0

These are synthetic benchmark results, not a production-wide quality claim.

Closes #478


Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor / cleanup
  • Other

PR Quality Checklist

  • PR title follows an allowed format
  • Changes are limited in scope and easy to review
  • Documentation updated where applicable; no public API or configuration signature changed
  • No breaking changes
  • Related issue linked

Optional

  • Screenshots or examples added; not applicable
  • Edge cases considered
  • Follow-up tasks mentioned

Validation

Passed from a clean worktree containing only this commit:

  • uv lock --locked
  • uv run pre-commit run -a
  • uv run mypy
  • uv run deptry src
  • full pytest suite with coverage

Caveats

  • duplicate ref_id values inside one scope return all distinct matching items
  • reference-aware end-to-end tests use the in-memory backend
  • existing SQLite and PostgreSQL repository implementations are unchanged
  • benchmark results are deterministic synthetic measurements only

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Reference-aware RAG retrieval from category citations

1 participant