Merge pull request #24 from nickroci/experiment/uv-tool-install

Experiment/uv tool install

Author: nickroci

.claude-plugin/marketplace.json

@@ -0,0 +1,11 @@
+{
+  "name": "ultan",
+  "owner": { "name": "Nicholas Holden" },
+  "plugins": [
+    {
+      "name": "ultan",
+      "source": "./",
+      "description": "Local, brain-inspired markdown memory for Claude Code — a curator pair gates writes by surprise, three retrieval tiers surface what matters, all on your disk."
+    }
+  ]
+}

.claude-plugin/plugin.json

@@ -0,0 +1,19 @@
+{
+  "name": "ultan",
+  "version": "0.2.0",
+  "description": "Local, brain-inspired markdown memory for Claude Code — a curator pair gates writes by surprise, three retrieval tiers surface what matters, all on your disk.",
+  "author": { "name": "Nicholas Holden", "url": "https://github.com/nickroci" },
+  "homepage": "https://github.com/nickroci/ultan",
+  "license": "MIT",
+  "mcpServers": {
+    "ultan": {
+      "command": "uvx",
+      "args": [
+        "--from",
+        "git+https://github.com/nickroci/ultan@main",
+        "ultan",
+        "mcp"
+      ]
+    }
+  }
+}

.github/workflows/ci.yml

@@ -17,12 +17,18 @@ jobs:
       fail-fast: false
       matrix:
         include:
+          # The root `ultan` plugin package (the runtime) and tools/ultan were
+          # previously unguarded — added so quality runs on every package.
+          - package: .
+            python-version: "3.12"
           - package: daemon
             python-version: "3.10"
           - package: tools/search
             python-version: "3.10"
           - package: src
             python-version: "3.12"
+          - package: tools/ultan
+            python-version: "3.12"
     defaults:
       run:
         working-directory: ${{ matrix.package }}
@@ -32,7 +38,10 @@ jobs:
       - uses: astral-sh/setup-uv@v6
         with:
           enable-cache: true
-          cache-dependency-glob: ${{ matrix.package }}/uv.lock
+          # The workspace root uv.lock is the only lockfile — member dirs
+          # resolve against it (`uv sync --locked` from a member validates
+          # the root lock), so it is the correct cache key for every job.
+          cache-dependency-glob: uv.lock
 
       - name: Install dependencies
         run: uv sync --locked

.pre-commit-config.yaml

@@ -156,3 +156,11 @@ repos:
         files: ^src/
         pass_filenames: false
         stages: [pre-push]
+
+      - id: pytest-root
+        name: pytest (root ultan package)
+        language: system
+        entry: bash -c 'uv run --frozen pytest'
+        files: ^(ultan/|tests/|pyproject\.toml)
+        pass_filenames: false
+        stages: [pre-push]

README.md

@@ -94,33 +94,58 @@ The deliberate choice: deterministic-and-cheap text retrieval at always-on Tier
 
 ## Quick start
 
-```bash
-# 1. Sync the daemon's deps (uv-managed)
-cd daemon && uv sync --group dev
+Ultan installs as a native **Claude Code plugin** — no editing `settings.json`, no
+daemon to babysit. You just need [`uv`](https://docs.astral.sh/uv) on your `PATH`; the
+plugin uses it to provision Ultan's runtime on first use.
 
-# 2. Sync the search CLI (separate venv, shared BM25 implementation).
-#    Pulls in sentence-transformers + einops; HuggingFace model weights
-#    are downloaded on first daemon start (see step 4 below).
-cd ../tools/search && uv sync
+Inside Claude Code:
 
-# 3. Install the slash commands and hooks
-#    - /ultan, /ultan-install, /ultan-advisor live at ~/.claude/commands/
-#    - `/ultan-install` writes hooks into ~/.claude/settings.json (GLOBAL — every
-#      Claude Code project). One daemon per machine serves the whole library
-#      across every repo, so global is the recommended default.
-#    - `/ultan-install --project` if you'd rather scope to one repo only.
+```text
+/plugin marketplace add nickroci/ultan
+/plugin install ultan@ultan      # choose "user" scope to enable it in every project
+/reload-plugins                  # load the hooks + MCP into the running session
+```
 
-# 4. Start the daemon (foreground; logs to ~/.agent-mem/daemon.log). One per
-#    machine — it listens on ~/.agent-mem/priming.sock and answers Tier-1
-#    priming requests from every hook on every project.
-cd /path/to/ultan/daemon && uv run agent-mem-daemon -v
-#    (nohup, tmux, or a launchd plist if you want it persistent — auto-supervision
-#     not yet implemented.)
+> ⏳ **`/plugin install` looks frozen — it isn't.** Claude Code shows **no progress
+> while it downloads the plugin, which can take a minute**. Don't cancel; the
+> "plugin changed" confirmation appears when it's done. After that, the first session
+> provisions Ultan's retrieval stack (torch + the embedding/rerank models — a few
+> hundred MB) in the background, and you can keep working while it finishes.
 
-# 5. Open Claude Code in any project (no per-project setup needed once the
-#    hooks are global) and work normally. Entries land under
-#    ~/.agent-mem/knowledge/ as the Scholar approves them.
-```
+That's it. Skills and slash commands hot-load the instant you install; `/reload-plugins`
+pulls in the hooks and the MCP server. **A full Claude Code restart is not required** — a
+fresh session also works, but you don't need one.
+
+> 🩺 **Wondering what it's doing? Ask Claude to run `ultan doctor`.** It reports
+> whether the background install is still running, the daemon's state (warming /
+> healthy / idle), priming latency, and capture freshness — at any stage, even
+> mid-install.
+
+On first use a `SessionStart` hook provisions that retrieval stack into the plugin's
+private storage **in the background**. Until it finishes, priming falls back to a fast
+lexical scan; after that the daemon **lazy-starts on demand**. Models download anonymously
+from HuggingFace — see *First-start expectations* below.
+
+You now have:
+
+- `/ultan <text>` to save a memory, `/ultan-advisor <question>` to consult the library
+- the `ultan-search` skill and the `ultan_recall` MCP tool
+- automatic priming on every prompt
+- everything under `~/.agent-mem/knowledge/` as plain markdown — local, no cloud, no telemetry
+
+> **Don't also run `/ultan-install`.** It wires a *second* copy of the hooks into
+> `settings.json`; with the plugin installed, every hook would then fire twice.
+> `/ultan-install` is only for the from-source path — see *Development setup* below.
+
+> **Already running Ultan from source?** If you previously ran `/ultan-install`, remove
+> its hooks *before* installing the plugin or every event will be captured twice:
+> open `~/.claude/settings.json` (or the project's `.claude/settings.json` if you used
+> `--project`) and delete every `hooks` entry whose command points into your agent-mem
+> checkout. Stop any manually started daemon too — the plugin's daemon lazy-starts on
+> its own. Your library needs no migration: `~/.agent-mem/` is shared by both setups.
+
+> The plugin currently installs from the `main` branch (`@main`) and will move to a
+> tagged release before wider promotion.
 
 ### Where your memories live
 
@@ -185,6 +210,41 @@ To save a memory explicitly: `/ultan never deploy to prod without my explicit OK
 
 To ask before asking the user: `/ultan-advisor should I use respx or hand-roll an httpx mock?`.
 
+## Development setup
+
+For hacking on Ultan itself, or running from source without the plugin. This is the
+manual path the plugin automates — you wire the hooks yourself and run the daemon
+directly from the repo.
+
+```bash
+# 1. Sync the daemon's deps (uv-managed)
+cd daemon && uv sync --group dev
+
+# 2. Sync the search CLI (separate venv, shared BM25 implementation).
+#    Pulls in sentence-transformers + einops; HuggingFace model weights
+#    are downloaded on first daemon start (see step 4 below).
+cd ../tools/search && uv sync
+
+# 3. Install the slash commands and hooks
+#    - /ultan, /ultan-install, /ultan-advisor live at ~/.claude/commands/
+#    - `/ultan-install` writes hooks into ~/.claude/settings.json (GLOBAL — every
+#      Claude Code project). One daemon per machine serves the whole library
+#      across every repo, so global is the recommended default.
+#    - `/ultan-install --project` if you'd rather scope to one repo only.
+#    NOTE: don't run this if you've installed the plugin — the hooks would double-fire.
+
+# 4. Start the daemon (foreground; logs to ~/.agent-mem/daemon.log). One per
+#    machine — it listens on ~/.agent-mem/priming.sock and answers Tier-1
+#    priming requests from every hook on every project.
+cd /path/to/ultan/daemon && uv run agent-mem-daemon -v
+#    (nohup, tmux, or a launchd plist if you want it persistent — auto-supervision
+#     not yet implemented.)
+
+# 5. Open Claude Code in any project (no per-project setup needed once the
+#    hooks are global) and work normally. Entries land under
+#    ~/.agent-mem/knowledge/ as the Scholar approves them.
+```
+
 ---
 
 ## How it works (the short version)
@@ -247,7 +307,7 @@ agent-mem/
     pyproject.toml      ← uv-managed
   src/                  ← Phase-0 hook layer (forked from claude-memory-compiler)
     hooks/              ← UserPromptSubmit, PostToolUse, Stop, ...
-    scripts/            ← compile / flush / lint / query
+    scripts/            ← flush / lint / query
     AGENTS.md           ← entry-schema reference
   tools/
     ultan/              ← /ultan, /ultan-install, /ultan-advisor scripts

bin/ultan

@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+# Plugin-root bin/ is added to the Bash tool's PATH while the plugin is
+# enabled — this is what lets /ultan, /ultan-advisor, and `ultan doctor` work
+# without the user installing anything on their own PATH. The real binary
+# lives in the plugin's private data dir (installed by ensure-ultan.sh).
+# CLAUDE_PLUGIN_DATA is not set for command-invoked bash, so fall back to the
+# default data location for this plugin+marketplace ("ultan" @ "ultan").
+DATA="${CLAUDE_PLUGIN_DATA:-$HOME/.claude/plugins/data/ultan-ultan}"
+REAL="$DATA/bin/ultan"
+if [ -x "$REAL" ]; then
+  exec "$REAL" "$@"
+fi
+
+# No binary yet — report WHERE the install stands instead of a bare error, so
+# `ultan doctor` answers "is it still installing?" even before the tool env
+# exists. (The full doctor lives in the installed package; this is the
+# pre-install half.)
+LOG="$DATA/install.log"
+PIDFILE="$DATA/.install.pid"
+if [ -f "$PIDFILE" ] && kill -0 "$(cat "$PIDFILE" 2>/dev/null)" 2>/dev/null; then
+  echo "ultan: STILL INSTALLING — the background 'uv tool install' is running." >&2
+  [ -f "$LOG" ] && echo "ultan: progress: $(tail -n 1 "$LOG" 2>/dev/null | cut -c1-120)" >&2
+  echo "ultan: first install downloads torch (minutes on a cold cache); try again shortly." >&2
+  exit 1
+fi
+if [ -f "$DATA/.install.lock" ]; then
+  echo "ultan: an install lock exists but no installer process is running —" >&2
+  echo "ultan: it either just started or died mid-install." >&2
+  [ -f "$LOG" ] && echo "ultan: last log line: $(tail -n 1 "$LOG" 2>/dev/null | cut -c1-120)" >&2
+  echo "ultan: a new Claude Code session retries automatically (stale locks clear after 30 min)." >&2
+  exit 1
+fi
+if [ -f "$LOG" ]; then
+  echo "ultan: NOT INSTALLED — the last install attempt ended without producing a binary." >&2
+  echo "ultan: install.log tail:" >&2
+  tail -n 3 "$LOG" 2>/dev/null | sed 's/^/ultan:   /' >&2
+  echo "ultan: a new Claude Code session retries automatically." >&2
+  exit 1
+fi
+echo "ultan: not provisioned yet — the plugin installs Ultan in the background at" >&2
+echo "ultan: session start (progress lands in $DATA/install.log)." >&2
+echo "ultan: if this is a fresh install, start a new session or wait a minute." >&2
+exit 1

commands/ultan-advisor.md

@@ -0,0 +1,25 @@
+---
+description: Consult Ultan's stored preferences before asking the user a preference-shaped question. Usage /ultan-advisor <the question you were going to ask>
+argument-hint: <the question or taste/convention decision>
+allowed-tools: Bash(ultan advisor:*)
+disable-model-invocation: true
+---
+
+Consult the user's Ultan memory library before asking them this question — they
+may have already answered it in a previous session. Run:
+
+```
+ultan advisor "$ARGUMENTS"
+```
+
+This runs the two-step advisor pipeline (Librarian finds relevant entries, then
+the Scholar writes a referenced markdown answer with `[[wikilink]]` citations).
+It can take 30-60s and is read-only — it never writes to the library. Use a
+generous Bash timeout (at least 120s).
+
+When it returns:
+- If the library has a relevant stored preference, follow it and tell the user
+  what Ultan already remembers, citing the `[[wikilink]]`. Don't re-ask what
+  they've already answered.
+- If the library is silent on the question, say so plainly, then proceed and ask
+  the user directly.

commands/ultan.md

@@ -0,0 +1,14 @@
+---
+description: Save a memory to Ultan (cross-session agent memory). Usage /ultan <text>
+argument-hint: <the memory text to remember>
+allowed-tools: Bash(ultan remember:*)
+disable-model-invocation: true
+---
+
+The user wants to commit something to long-term Ultan memory. Run it now:
+
+!`ultan remember "$ARGUMENTS"`
+
+The command above appends a user-asserted event to the daemon's queue; the
+Librarian picks it up and files it. Report the confirmation line ("queued for
+librarian …") back to the user in one short sentence. Do not do anything else.

daemon/agent_mem_daemon/__main__.py

@@ -335,6 +335,13 @@ def run(args: argparse.Namespace) -> int:
     logfile = args.log_file or log_path()
     pidfile = args.pid_file or pid_path()
 
+    # Claim the PID file BEFORE configure_logging: configure_logging rotates
+    # daemon.log when it grows past the cap, so a rejected duplicate spawn
+    # (acquire_pid_file → SystemExit(2)) must not touch the live daemon's log
+    # files on its way out. acquire_pid_file writes only to raw stderr, which
+    # the spawner tees to daemon-spawn.log (see ultan/_daemon.py).
+    acquire_pid_file(pidfile)
+
     log = configure_logging(
         logfile,
         level=logging.DEBUG if args.verbose else logging.INFO,
@@ -345,8 +352,6 @@ def run(args: argparse.Namespace) -> int:
     log.info("  log file:    %s", logfile)
     log.info("  pid file:    %s", pidfile)
 
-    acquire_pid_file(pidfile)
-
     # Verify indexes are in sync with the markdown source of truth.
     # Rebuilds on drift (manual edits, git pull from another machine,
     # restore from backup); logs "ready (N docs)" otherwise. Failures

daemon/agent_mem_daemon/decay.py

@@ -120,6 +120,17 @@ def _read_and_parse_frontmatter(
     return fm, m.group(1), body
 
 
+def _lacks_frontmatter_fence(entry_path: Path) -> bool:
+    """True when the file is readable but has no ``---`` frontmatter fence —
+    i.e. it is a catalog file (folder README), not a malformed entry. An
+    unreadable file returns False so the caller counts it as a real error."""
+    try:
+        text = entry_path.read_text(encoding="utf-8")
+    except OSError:
+        return False
+    return _FRONTMATTER_HEAD_RE.match(text) is None
+
+
 def _atomic_write_entry(entry_path: Path, fm: Dict[str, Any], body: str) -> bool:
     """Serialise ``fm`` back to YAML and rewrite the entry atomically.
     Returns True on success, False on any I/O / YAML failure."""
@@ -449,10 +460,17 @@ def _archive_entry(
 @dataclass
 class SweepResult:
     """Summary of one sweep pass — what got moved, what got skipped,
-    what failed. Returned by :func:`run_sweep` and logged at INFO."""
+    what failed. Returned by :func:`run_sweep` and logged at INFO.
+
+    ``skipped`` counts files with no frontmatter fence at all — folder
+    ``README.md`` catalogs and the like. Those are catalog content, not
+    decayable entries, and MUST NOT count as ``errored``: a library of a
+    few hundred entries carries ~25% READMEs, and lumping them into the
+    error count buries real parse failures in constant noise."""
 
     archived: int = 0
     kept: int = 0
+    skipped: int = 0
     errored: int = 0
     archived_paths: List[str] = field(default_factory=lambda: [])
 
@@ -512,7 +530,13 @@ def run_sweep(
         try:
             parsed = _read_and_parse_frontmatter(entry_path)
             if parsed is None:
-                result.errored += 1
+                if _lacks_frontmatter_fence(entry_path):
+                    # Catalog file (folder README etc.) — not a decayable
+                    # entry and not an error.
+                    result.skipped += 1
+                else:
+                    log.warning("decay.run_sweep: unparseable frontmatter in %s", entry_path)
+                    result.errored += 1
                 continue
             fm, _raw, body = parsed
             if not _is_eligible_for_archive(fm, today=today):
@@ -535,9 +559,10 @@ def run_sweep(
 
     write_last_sweep_at(when)
     log.info(
-        "decay.sweep: archived=%d kept=%d errored=%d",
+        "decay.sweep: archived=%d kept=%d skipped=%d errored=%d",
         result.archived,
         result.kept,
+        result.skipped,
         result.errored,
     )
     return result