feat: add Milvus vector index provider

Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>

Author: zc277584121

README.md

@@ -269,6 +269,15 @@ cd tests
 uv run python test_postgres.py
 ```
 
+**Run with Milvus Lite as an external vector index:**
+```bash
+uv sync --extra milvus
+export OPENAI_API_KEY=your_key
+uv run python examples/milvus_vector_index.py
+```
+
+The Milvus integration currently wires `vector_index.provider="milvus"` to the `inmemory` metadata store. Use `uri` to point the same configuration at Milvus Lite, a Milvus server, or Zilliz Cloud.
+
 ---
 
 ### Custom LLM and Embedding Providers

docs/adr/0008-external-vector-index.md

@@ -0,0 +1,40 @@
+# ADR 0008: External Vector Index Alongside Metadata Store
+
+- Status: Accepted
+- Date: 2026-06-30
+
+## Context
+
+ADR 0002 locked in repository-based storage with backend-aware vector search: the `inmemory` and `sqlite` backends do brute-force cosine, and `postgres` can use pgvector. Two footprints are not well served by that design:
+
+- Multi-million-vector workloads, where brute-force is too slow and pgvector requires keeping Postgres hot enough to hold the index.
+- Managed deployments that already standardise on a dedicated vector service (Milvus, Zilliz Cloud) and want memU to reuse it.
+
+`settings.py` already separates `metadata_store` from `vector_index` in configuration, but every shipped backend kept vectors inside its own tables. The separation was not realised in code.
+
+## Decision
+
+Introduce a `VectorIndex` protocol that memory repositories can delegate to when an external index is configured. The first implementation is `MilvusVectorIndex`, targeting Milvus Lite (zero-dep local), self-hosted Milvus, and Zilliz Cloud behind a single `uri`/`token` config.
+
+- `VectorIndex` lives in `memu/database/vector_index/` and exposes `upsert`, `delete`, `delete_many`, `search`, `close`.
+- `build_vector_index` in the factory constructs the provider based on `VectorIndexConfig.provider`.
+- The in-memory metadata backend receives the vector index through its builder and mirrors mutations to it on create / update / delete / clear.
+- Search routes through the vector index when present; salience ranking stays local because it needs per-item reinforcement and recency factors the index does not persist.
+
+Initial scope: the `inmemory` metadata backend is wired to `MilvusVectorIndex`. `sqlite` and `postgres` are planned follow-ups; their existing (brute-force / pgvector) paths remain the default so the change is additive. Until those follow-ups land, memU rejects `vector_index.provider="milvus"` with non-`inmemory` metadata stores to avoid silently ignoring the configured external index.
+
+## Consequences
+
+Positive:
+
+- memU can scale vector similarity independently of the metadata store.
+- Users on Zilliz Cloud or a shared Milvus cluster can reuse that infrastructure.
+- Milvus Lite keeps the zero-setup local story (single file, no service) that `inmemory` already promises.
+
+Negative:
+
+- Two systems of record mean write amplification and a consistency window between the metadata store and the vector index.
+- Scope fields are replicated into Milvus as dynamic fields, increasing storage vs. a single SQL column.
+- `pymilvus` / `milvus-lite` are optional and gated behind the `milvus` extra; users picking this path accept that dependency.
+- Milvus Lite 3.x uses a newer local storage format than the earlier 2.x line, so users upgrading existing local `.db` files should create a new Milvus Lite database and re-ingest their vectors.
+- Milvus Lite 3.0 still imports `pkg_resources` on Python 3.13, so the `milvus` extra pins `setuptools<81` until that dependency is removed upstream.

docs/adr/README.md

@@ -7,3 +7,4 @@
 - [0005: Extract Embedding into a Dedicated Package, Fully Decoupled from Chat Clients](0005-dedicated-embedding-package.md)
 - [0006: Promote Skills to a First-Class `RecallFile` Track, Generated Inside Memorize](0006-from-memory-item-category-to-tracked-workspace-memorization.md)
 - [0007: Split Memorize/Retrieve into Three Independent Lines on a Layered Wiki-Graph Kernel](0007-three-independent-memory-lines-wiki-graph.md)
+- [0008: External Vector Index Alongside Metadata Store](0008-external-vector-index.md)

docs/architecture.md

@@ -219,8 +219,15 @@ stored vectors.)
 - `postgres`: SQLModel persistence with pgvector support (when enabled), local fallback ranking when needed
 
 `DatabaseConfig` auto-derives `vector_index`: `pgvector` for postgres, `bruteforce`
-otherwise. For Postgres, startup runs migration bootstrap and attempts
-`CREATE EXTENSION IF NOT EXISTS vector` in `ddl_mode="create"`.
+otherwise. Users can also set `database_config.vector_index` explicitly to select
+vector-search behavior:
+
+- `bruteforce`: local cosine search in the metadata backend
+- `pgvector`: Postgres-native vector search when the metadata store is `postgres`
+- `milvus`: external Milvus / Milvus Lite / Zilliz Cloud vector index for the `inmemory` metadata store
+- `none`: reserved for disabling external vector-index construction
+
+For Postgres, startup runs migration bootstrap and attempts `CREATE EXTENSION IF NOT EXISTS vector` in `ddl_mode="create"`.
 
 ### Track column and scope model
 

docs/milvus.md

@@ -0,0 +1,86 @@
+# Milvus Vector Index
+
+memU can route in-memory metadata-store similarity search to [Milvus](https://milvus.io/) as an external vector index. This is useful when:
+
+- **Scale**: you have more vectors than the brute-force fallback can handle at latency budget.
+- **Managed deployments**: you want to offload vector infrastructure to Zilliz Cloud or a self-hosted Milvus cluster.
+- **Zero-setup local dev**: Milvus Lite runs as a local file (`./milvus.db`) with no external service.
+
+## Install
+
+```bash
+uv sync --extra milvus
+```
+
+> The `milvus` extra pulls `pymilvus`, `milvus-lite` and — on Python 3.13 — a `setuptools<81` compatibility pin because Milvus Lite 3.0 still imports `pkg_resources`. New local Milvus Lite files should use the 3.x storage format; older 2.x `.db` files are not expected to be reusable after upgrading to Milvus Lite 3.x.
+
+## Quick Start (Milvus Lite)
+
+```python
+from memu.app import MemoryService
+
+service = MemoryService(
+    llm_profiles={"default": {"api_key": "your-api-key"}},
+    database_config={
+        "metadata_store": {"provider": "inmemory"},
+        "vector_index": {"provider": "milvus"},
+    },
+)
+```
+
+With the default configuration the index is persisted to `./milvus.db` using Milvus Lite — no Docker, no separate process.
+
+## Targeting a Milvus Server
+
+```python
+database_config = {
+    "metadata_store": {"provider": "inmemory"},
+    "vector_index": {
+        "provider": "milvus",
+        "uri": "http://localhost:19530",
+        "collection_name": "memu_prod",
+    },
+}
+```
+
+## Targeting Zilliz Cloud
+
+```python
+import os
+
+database_config = {
+    "metadata_store": {"provider": "inmemory"},
+    "vector_index": {
+        "provider": "milvus",
+        "uri": os.environ["ZILLIZ_URI"],
+        "token": os.environ["ZILLIZ_TOKEN"],
+        "collection_name": "memu_prod",
+    },
+}
+```
+
+## Configuration
+
+| Field | Default | Notes |
+| --- | --- | --- |
+| `provider` | — | Must be `"milvus"` to enable this index. |
+| `uri` | `"./milvus.db"` | File path runs Milvus Lite; `http(s)://host:port` targets a Milvus server; a Zilliz Cloud endpoint targets the managed service. |
+| `token` | `None` | Auth token for Zilliz Cloud or a secured Milvus server. |
+| `db_name` | `None` | Optional Milvus database name. |
+| `collection_name` | `"memu_memory_items"` | Name of the Milvus collection that holds memory vectors. |
+| `dim` | `None` | Embedding dimension. Inferred from the first upsert when omitted. |
+| `consistency_level` | `None` | Optional Milvus collection consistency level (`"Strong"`, `"Session"`, `"Bounded"`, or `"Eventually"`). Uses the server default when omitted. |
+
+## Supported Combinations
+
+memU keeps metadata records (summary, categories, scope, reinforcement stats, ...) in the metadata store and mirrors embeddings into Milvus on create / update / delete.
+
+- **Available now**: `inmemory` metadata store + Milvus vector index (feature-complete).
+- **Not wired yet**: `sqlite` and `postgres` metadata stores reject `vector_index.provider="milvus"` rather than silently ignoring it. See `docs/adr/0008-external-vector-index.md` for the rollout plan.
+
+## How Search Works
+
+1. `create_item` / `update_item` mirror the embedding (and scope fields such as `user_id`, `agent_id`) into the configured Milvus collection.
+2. `vector_search_items` forwards the query vector to Milvus using a COSINE AUTOINDEX. Scope filters (e.g. `where={"user_id": "u1"}`) are translated into Milvus boolean expressions on dynamic fields.
+3. Milvus returns `(id, score)` pairs; the metadata store resolves them back to full memory records.
+4. Salience ranking (`ranking="salience"`) still runs locally because it needs per-item reinforcement and recency factors that the vector index does not store.

examples/milvus_vector_index.py

@@ -0,0 +1,72 @@
+"""Minimal end-to-end example: memU backed by a Milvus vector index.
+
+Uses the ``inmemory`` metadata store and Milvus Lite (a single ``milvus.db``
+file) so it runs with zero external services. Point ``uri`` at a Milvus server
+URL or a Zilliz Cloud endpoint to scale to production.
+
+Run:
+
+    uv sync --extra milvus
+    export OPENAI_API_KEY=sk-...
+    uv run python examples/milvus_vector_index.py
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+from pathlib import Path
+
+from memu.app import MemoryService
+
+
+async def main() -> None:
+    api_key = os.environ.get("OPENAI_API_KEY")
+    if not api_key:
+        msg = "Set OPENAI_API_KEY before running this example."
+        raise SystemExit(msg)
+
+    file_path = os.path.abspath("examples/resources/conversations/conv1.json")
+    if not Path(file_path).exists():
+        msg = (
+            f"Example conversation not found at {file_path}. "
+            "Run from the repository root so examples/resources/conversations/conv1.json is available."
+        )
+        raise SystemExit(msg)
+
+    service = MemoryService(
+        llm_profiles={"default": {"api_key": api_key}},
+        database_config={
+            "metadata_store": {"provider": "inmemory"},
+            "vector_index": {
+                "provider": "milvus",
+                # Local Milvus Lite file. Swap for "http://host:19530" or a
+                # Zilliz Cloud endpoint to target a real deployment.
+                "uri": "./milvus.db",
+                "collection_name": "memu_example",
+            },
+        },
+        retrieve_config={"method": "rag"},
+    )
+
+    print("[memU + Milvus] Memorizing example conversation...")
+    memory = await service.memorize(
+        resource_url=file_path,
+        modality="conversation",
+        user={"user_id": "demo-user"},
+    )
+    for cat in memory.get("categories", []):
+        print(f"  - {cat.get('name')}: {(cat.get('summary') or '')[:80]}...")
+
+    queries = [
+        {"role": "user", "content": {"text": "What do you know about my preferences?"}},
+    ]
+    result = await service.retrieve(queries=queries, where={"user_id": "demo-user"})
+
+    print("\n[memU + Milvus] Retrieved items:")
+    for item in result.get("items", [])[:5]:
+        print(f"  - [{item.get('memory_type')}] {(item.get('summary') or '')[:100]}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

pyproject.toml

@@ -68,6 +68,12 @@ test = [
 
 [project.optional-dependencies]
 postgres = ["pgvector>=0.3.4", "sqlalchemy[postgresql-psycopgbinary]>=2.0.36"]
+milvus = [
+    "pymilvus>=3.0.0,<4.0.0",
+    "milvus-lite>=3.0,<4.0",
+    # milvus-lite 3.0 still imports pkg_resources on Python 3.13.
+    "setuptools<81; python_version >= '3.13'",
+]
 langgraph = ["langgraph>=0.0.10", "langchain-core>=0.1.0"]
 claude = ["claude-agent-sdk>=0.1.24"]
 lazyllm = ["lazyllm>=0.7.3"]
@@ -84,7 +90,7 @@ memu = "memu.cli:main"
 
 [tool.deptry.per_rule_ignores]
 # Optional dependencies used in examples/ or imported lazily behind extras.
-DEP002 = ["claude-agent-sdk", "markitdown"]
+DEP002 = ["claude-agent-sdk", "markitdown", "milvus-lite", "setuptools"]
 
 [tool.mypy]
 files = ["src", "tests"]
@@ -116,6 +122,10 @@ ignore_missing_imports = true
 module = ["markitdown", "markitdown.*"]
 ignore_missing_imports = true
 
+[[tool.mypy.overrides]]
+module = ["pymilvus.*"]
+ignore_missing_imports = true
+
 [tool.ruff]
 target-version = "py313"
 line-length = 120

src/memu/app/settings.py

@@ -576,8 +576,23 @@ class MetadataStoreConfig(BaseModel):
 
 
 class VectorIndexConfig(BaseModel):
-    provider: Annotated[Literal["bruteforce", "pgvector", "none"], Normalize] = "bruteforce"
+    provider: Annotated[Literal["bruteforce", "pgvector", "milvus", "none"], Normalize] = "bruteforce"
     dsn: str | None = Field(default=None, description="Postgres connection string when provider=pgvector.")
+    uri: str | None = Field(
+        default=None,
+        description=(
+            "Milvus URI when provider=milvus. A file path such as './milvus.db' uses Milvus Lite; "
+            "http(s)://host:port targets Milvus server or Zilliz Cloud."
+        ),
+    )
+    token: str | None = Field(default=None, description="Milvus/Zilliz auth token, if required.")
+    db_name: str | None = Field(default=None, description="Milvus database name, if required.")
+    collection_name: str = Field(default="memu_memory_items", description="Milvus collection name.")
+    dim: int | None = Field(default=None, description="Embedding dimension for Milvus collection creation.")
+    consistency_level: Literal["Strong", "Session", "Bounded", "Eventually"] | None = Field(
+        default=None,
+        description="Milvus collection consistency level. Uses the server default when omitted.",
+    )
 
 
 class DatabaseConfig(BaseModel):
@@ -592,3 +607,5 @@ def model_post_init(self, __context: Any) -> None:
                 self.vector_index = VectorIndexConfig(provider="bruteforce")
         elif self.vector_index.provider == "pgvector" and self.vector_index.dsn is None:
             self.vector_index = self.vector_index.model_copy(update={"dsn": self.metadata_store.dsn})
+        elif self.vector_index.provider == "milvus" and self.vector_index.uri is None:
+            self.vector_index = self.vector_index.model_copy(update={"uri": "./milvus.db"})

src/memu/database/factory.py

@@ -7,6 +7,7 @@
 from memu.app.settings import DatabaseConfig
 from memu.database.inmemory import build_inmemory_database
 from memu.database.interfaces import Database
+from memu.database.vector_index import build_vector_index
 
 if TYPE_CHECKING:
     pass
@@ -26,8 +27,14 @@ def build_database(
         - "sqlite": SQLite file-based storage (lightweight, portable)
     """
     provider = config.metadata_store.provider
+    vector_provider = config.vector_index.provider if config.vector_index else None
+    if vector_provider == "milvus" and provider != "inmemory":
+        msg = "vector_index provider 'milvus' currently supports metadata_store provider 'inmemory' only"
+        raise ValueError(msg)
+
     if provider == "inmemory":
-        return build_inmemory_database(config=config, user_model=user_model)
+        vector_index = build_vector_index(config.vector_index)
+        return build_inmemory_database(config=config, user_model=user_model, vector_index=vector_index)
     elif provider == "postgres":
         # Lazy import to avoid requiring pgvector when not using postgres
         from memu.database.postgres import build_postgres_database

src/memu/database/inmemory/__init__.py

@@ -5,12 +5,14 @@
 from memu.app.settings import DatabaseConfig
 from memu.database.inmemory.models import build_inmemory_models
 from memu.database.inmemory.repo import InMemoryStore
+from memu.database.vector_index.interfaces import VectorIndex
 
 
 def build_inmemory_database(
     *,
     config: DatabaseConfig,
     user_model: type[BaseModel],
+    vector_index: VectorIndex | None = None,
 ) -> InMemoryStore:
     (
         resource_model,
@@ -28,6 +30,7 @@ def build_inmemory_database(
         recall_file_entry_model=recall_file_entry_model,
         recall_file_resource_model=recall_file_resource_model,
         recall_file_segment_model=recall_file_segment_model,
+        vector_index=vector_index,
     )