Skip to content

Commit ff6a3e4

Browse files
authored
fix(sync): bound daemon memory and watcher recovery (#1154)
## Why On macOS, recursive kqueue watching kept state proportional to every file in a long-lived archive. Passive daemon memory therefore grew with archive size, while broad polling and recovery scans could keep CPU and I/O active. ## What changed - Active recursive Darwin roots use native FSEvents, so passive watcher state scales with configured roots rather than stored files. Callback queues retain the existing path and byte limits. - Discovery and authoritative reconciliation stream through disk-backed pages. Provider enumeration, duplicate selection, ordered aggregation, Trae workspace discovery, and Hermes profile/member discovery no longer require archive-sized Go maps or repeated value concatenation. - Watcher overflow, directory lifecycle events, fallback coverage, and polling retain bounded retry scopes where correctness permits. Non-Darwin fsnotify coverage applies the same watch budget to newly created and moved-in subtrees. - Missing-source deletion is conservative and recoverable. Only complete machine-local provider scopes establish deletion proof, including virtual members in multi-session containers, and scheduled reconciliation defers scopes whose physical roots are unavailable. Tombstones durably invalidate base and hash-qualified skip entries before hiding a session, and audit workers refresh the daemon cache so byte-identical restored files are parsed and revived across workers and restarts. - Source revival replaces stale content before becoming visible, and provider-specific discovery preserves canonical identities across symlinks, overlapping roots, expanded Hermes profile archives, and mixed filesystem/SQLite layouts. ## Failure behavior Root-local discovery failures may still yield valid sessions and do not block later configured roots. Those partial candidates can be stored, but the failed provider scope cannot add baselines or tombstone missing sessions; independent healthy providers remain authoritative. Callback, cancellation, temporary-store, and cleanup failures stop immediately, and the watcher retains the affected roots for retry. True event loss or ownerless recovery can still require global authoritative reconciliation. Runtime watcher fallback and low-frequency reconciliation remain available for filesystems without reliable native events. Focused macOS FSEvents CI runs only on trusted pushes to `main` through the self-hosted runner group. Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>
1 parent 69e6ef6 commit ff6a3e4

231 files changed

Lines changed: 46498 additions & 2984 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: CI (macOS)
2+
3+
on:
4+
# Persistent self-hosted runners must never be selectable from a workflow
5+
# definition controlled by a pull request. The ci-runners group must allow
6+
# this exact workflow path only from refs/heads/main.
7+
push:
8+
branches: [main]
9+
10+
permissions:
11+
contents: read
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
jobs:
18+
test-macos-fsevents:
19+
name: Go Test (macOS FSEvents)
20+
runs-on:
21+
group: ci-runners
22+
labels: [self-hosted, macOS, ARM64]
23+
steps:
24+
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
25+
with:
26+
fetch-depth: 0
27+
persist-credentials: false
28+
29+
- uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
30+
with:
31+
go-version-file: go.mod
32+
33+
- name: Restore pricing snapshot
34+
run: go run ./internal/pricing/cmd/litellm-snapshot -restore
35+
36+
- name: Run focused macOS FSEvents tests
37+
run: CGO_ENABLED=1 go test -tags "fts5" ./internal/fsevents ./internal/sync ./internal/db ./cmd/agentsview -count=1 -timeout=20m

.github/workflows/desktop-macos-main.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
name: Desktop Artifacts (macOS)
22

33
on:
4-
# This workflow is the only workflow allowed to use the ci-runners group.
5-
# The organization runner-group policy must pin access to this path on main.
4+
# macOS self-hosted workflows are push-only for main. The organization
5+
# runner-group policy must pin access to their exact workflow paths on main.
66
push:
77
branches: [main]
88
paths:

0 commit comments

Comments
 (0)