Commit ff6a3e4
authored
fix(sync): bound daemon memory and watcher recovery (#1154)
## Why
On macOS, recursive kqueue watching kept state proportional to every file in a long-lived archive. Passive daemon memory therefore grew with archive size, while broad polling and recovery scans could keep CPU and I/O active.
## What changed
- Active recursive Darwin roots use native FSEvents, so passive watcher state scales with configured roots rather than stored files. Callback queues retain the existing path and byte limits.
- Discovery and authoritative reconciliation stream through disk-backed pages. Provider enumeration, duplicate selection, ordered aggregation, Trae workspace discovery, and Hermes profile/member discovery no longer require archive-sized Go maps or repeated value concatenation.
- Watcher overflow, directory lifecycle events, fallback coverage, and polling retain bounded retry scopes where correctness permits. Non-Darwin fsnotify coverage applies the same watch budget to newly created and moved-in subtrees.
- Missing-source deletion is conservative and recoverable. Only complete machine-local provider scopes establish deletion proof, including virtual members in multi-session containers, and scheduled reconciliation defers scopes whose physical roots are unavailable. Tombstones durably invalidate base and hash-qualified skip entries before hiding a session, and audit workers refresh the daemon cache so byte-identical restored files are parsed and revived across workers and restarts.
- Source revival replaces stale content before becoming visible, and provider-specific discovery preserves canonical identities across symlinks, overlapping roots, expanded Hermes profile archives, and mixed filesystem/SQLite layouts.
## Failure behavior
Root-local discovery failures may still yield valid sessions and do not block later configured roots. Those partial candidates can be stored, but the failed provider scope cannot add baselines or tombstone missing sessions; independent healthy providers remain authoritative. Callback, cancellation, temporary-store, and cleanup failures stop immediately, and the watcher retains the affected roots for retry.
True event loss or ownerless recovery can still require global authoritative reconciliation. Runtime watcher fallback and low-frequency reconciliation remain available for filesystems without reliable native events.
Focused macOS FSEvents CI runs only on trusted pushes to `main` through the self-hosted runner group.
Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>1 parent 69e6ef6 commit ff6a3e4
231 files changed
Lines changed: 46498 additions & 2984 deletions
File tree
- .github/workflows
- cmd/agentsview
- docs/internal
- frontend/src/lib/api
- internal
- db
- duckdb
- fsevents
- parser
- postgres
- server
- sync
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | | - | |
| 4 | + | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
0 commit comments