Skip to content

Commit f0942ab

Browse files
xkrogenmjacobs
andauthored
feat(sync): add machine-labeled session sources (#1170)
Add filesystem-only `[[session_sources]]` entries with `agent`, `dir`, and optional `machine`. Structured sources are additive to existing per-agent directory arrays, defaults, and environment variables; when roots overlap, the structured entry supplies the machine label. Configured paths keep the spelling the user wrote them in, and a separate cleaned comparison key collapses equivalent roots, so Windows-style entries survive a config round-trip. A session's machine is resolved from the configured root that produced it and is assigned once, when the session is first ingested. After that the label is immutable: ordinary periodic, watcher-driven, incremental, provider-backed, and single-session syncs all retain the stored value rather than adopting a newly configured one. `agentsview sync --full` preserves stored labels while rebuilding the archive; it is not a relabel operation. Changing a configured label therefore affects only sessions discovered afterward, which makes editing configuration safe rather than destructive. Filesystem labels do not namespace native session IDs. Because relabeling is no longer part of sync, this removes the machine-only update for trashed sessions, the all-machines baseline scan, and the ownership baseline index keyed without machine. Watch reconciliation still uses the baseline for the machine that originally admitted a session, so per-event work stays bounded by the changed batch. An earlier revision of this branch chased reattribution through each new writer, trash path, snapshot path, and reconciliation edge. That approach kept widening the persistent-archive surface and reintroducing consistency bugs, so it was replaced with the ingestion-time contract above. A one-shot relabel command is deliberately deferred: it needs its own invariants and recovery tests, and it is not required to ship labeled ingestion. The filesystem sync guide documents Git, rsync/file copy, and NFS/shared-mount topologies, including atomic publication, archive ownership, freshness, native-ID deduplication, label-change behavior, and when PostgreSQL is a better fit. `session_sources` intentionally rejects `s3://` roots. Existing Claude and Codex S3 configuration and behavior remain unchanged through their existing per-agent settings. Reviewers should start at `internal/sync/engine.go`, where machine resolution and the preserve-on-upsert rule live, then `internal/config/config.go` for parsing and path handling. `docs/superpowers/specs/` carries the design rationale. This branch contains no DuckDB changes. Mirror source-machine attribution already landed in #1302, and this PR neither duplicates nor modifies it. --------- Co-authored-by: Matthew Jacobs <mjacobs@apache.org>
1 parent e51f68b commit f0942ab

47 files changed

Lines changed: 3972 additions & 186 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,29 @@ March 2026), which shares that same database. *Kilo (legacy)* is the legacy
480480
RooCode-derived VS Code extension that wrote per-task JSON under
481481
`kilocode.kilo-code/tasks/`.
482482

483+
## Filesystem Session Sync
484+
485+
One primary AgentsView instance can ingest native agent session directories
486+
copied or mounted from other machines without PostgreSQL:
487+
488+
```toml
489+
[[session_sources]]
490+
agent = "copilot"
491+
dir = "/srv/session-archive/buildbox/copilot"
492+
machine = "buildbox"
493+
```
494+
495+
Structured sources are additive to existing `copilot_dirs`,
496+
`claude_project_dirs`, and other per-agent settings. They label sessions by
497+
source machine without namespacing native session IDs. Transport source session
498+
files only -- never copy `sessions.db` or its WAL files. Machine labels are
499+
captured at first ingestion; ordinary sync and `agentsview sync --full` preserve
500+
the stored label. Changing attribution for existing sessions is not currently
501+
supported.
502+
503+
See the [Filesystem Session Sync guide](https://agentsview.io/filesystem-sync/)
504+
for Git, rsync, shared-mount, freshness, and operational guidance.
505+
483506
## PostgreSQL Sync
484507

485508
Push session data to a shared PostgreSQL instance for team dashboards:
@@ -633,6 +656,7 @@ Full docs at **[agentsview.io](https://agentsview.io)**:
633656
[Usage Guide](https://agentsview.io/usage/) --
634657
[CLI Reference](https://agentsview.io/commands/) --
635658
[Configuration](https://agentsview.io/configuration/) --
659+
[Filesystem Sync](https://agentsview.io/filesystem-sync/) --
636660
[Architecture](https://agentsview.io/architecture/)
637661

638662
______________________________________________________________________

cmd/agentsview/archive_query_backend.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,6 +250,7 @@ func (b localArchiveQueryBackend) SessionUsage(
250250
if known && !b.skipFreshData {
251251
engine := sync.NewEngine(b.database, sync.EngineConfig{
252252
AgentDirs: b.cfg.AgentDirs,
253+
SourceMachines: b.cfg.SourceMachines,
253254
IncludeCwdPrefixes: b.cfg.SyncIncludeCwdPrefixes,
254255
Machine: b.cfg.LocalMachineName,
255256
BlockedResultCategories: b.cfg.ResultContentBlockedCategories,

cmd/agentsview/archive_write_backend.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -936,6 +936,7 @@ func (b *localArchiveWriteBackend) DuckDBPushWatch(
936936

937937
engine := syncpkg.NewEngine(b.database, syncpkg.EngineConfig{
938938
AgentDirs: b.appCfg.AgentDirs,
939+
SourceMachines: b.appCfg.SourceMachines,
939940
IncludeCwdPrefixes: b.appCfg.SyncIncludeCwdPrefixes,
940941
Machine: b.appCfg.LocalMachineName,
941942
BlockedResultCategories: b.appCfg.ResultContentBlockedCategories,
@@ -1053,6 +1054,7 @@ func (b *localArchiveWriteBackend) PGPushWatch(
10531054

10541055
engine := syncpkg.NewEngine(b.database, syncpkg.EngineConfig{
10551056
AgentDirs: b.appCfg.AgentDirs,
1057+
SourceMachines: b.appCfg.SourceMachines,
10561058
IncludeCwdPrefixes: b.appCfg.SyncIncludeCwdPrefixes,
10571059
Machine: b.appCfg.LocalMachineName,
10581060
BlockedResultCategories: b.appCfg.ResultContentBlockedCategories,

cmd/agentsview/main.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,7 @@ func runServe(cfg config.Config, opts serveOptions) {
296296
var onStartupReconciled func(sync.SyncStats, error)
297297
engine = sync.NewEngine(database, sync.EngineConfig{
298298
AgentDirs: cfg.AgentDirs,
299+
SourceMachines: cfg.SourceMachines,
299300
IncludeCwdPrefixes: cfg.SyncIncludeCwdPrefixes,
300301
Machine: cfg.LocalMachineName,
301302
BlockedResultCategories: cfg.ResultContentBlockedCategories,

cmd/agentsview/parse_diff.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,7 @@ func doParseDiff(cfg ParseDiffConfig) (failed bool) {
127127

128128
engine := sync.NewDiffEngine(database, sync.EngineConfig{
129129
AgentDirs: appCfg.AgentDirs,
130+
SourceMachines: appCfg.SourceMachines,
130131
IncludeCwdPrefixes: appCfg.SyncIncludeCwdPrefixes,
131132
Machine: appCfg.LocalMachineName,
132133
BlockedResultCategories: appCfg.ResultContentBlockedCategories,

cmd/agentsview/session_sync.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ func syncService(
7474
}
7575
engine := sync.NewEngine(d, sync.EngineConfig{
7676
AgentDirs: cfg.AgentDirs,
77+
SourceMachines: cfg.SourceMachines,
7778
IncludeCwdPrefixes: cfg.SyncIncludeCwdPrefixes,
7879
Machine: cfg.LocalMachineName,
7980
})

cmd/agentsview/sync.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -933,6 +933,7 @@ func coordinateLocalSync(
933933

934934
engine := sync.NewEngine(database, sync.EngineConfig{
935935
AgentDirs: appCfg.AgentDirs,
936+
SourceMachines: appCfg.SourceMachines,
936937
IncludeCwdPrefixes: appCfg.SyncIncludeCwdPrefixes,
937938
Machine: appCfg.LocalMachineName,
938939
BlockedResultCategories: appCfg.ResultContentBlockedCategories,

cmd/agentsview/sync_worker.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -360,6 +360,7 @@ func openWorkerWriteDB(cfg config.Config) (*db.DB, *writeOwnerLock, error) {
360360
func workerEngineConfig(cfg config.Config) sync.EngineConfig {
361361
return sync.EngineConfig{
362362
AgentDirs: cfg.AgentDirs,
363+
SourceMachines: cfg.SourceMachines,
363364
IncludeCwdPrefixes: cfg.SyncIncludeCwdPrefixes,
364365
Machine: cfg.LocalMachineName,
365366
BlockedResultCategories: cfg.ResultContentBlockedCategories,

cmd/agentsview/sync_worker_test.go

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,28 @@ func TestSyncWorkerStartupModeSyncsAndEmitsTerminalResult(t *testing.T) {
9999
"public SyncStats fields must survive the NDJSON protocol")
100100
}
101101

102+
func TestSyncWorkerStartupUsesConfiguredSourceMachine(t *testing.T) {
103+
cfg := testConfigWithClaudeFixture(t)
104+
claudeRoot := cfg.AgentDirs[parser.AgentClaude][0]
105+
cfg.SourceMachines = map[parser.AgentType]map[string]string{
106+
parser.AgentClaude: {claudeRoot: "archivebox"},
107+
}
108+
109+
var out bytes.Buffer
110+
require.NoError(t, runSyncWorker(cfg, "startup", &out))
111+
assert.Equal(t, "ok", decodeSingleResult(t, &out).Status)
112+
113+
database, err := db.OpenReadOnly(cfg.DBPath)
114+
require.NoError(t, err)
115+
defer database.Close()
116+
page, err := database.ListSessions(context.Background(), db.SessionFilter{})
117+
require.NoError(t, err)
118+
require.Len(t, page.Sessions, 3)
119+
for _, sess := range page.Sessions {
120+
assert.Equal(t, "archivebox", sess.Machine)
121+
}
122+
}
123+
102124
func TestSyncWorkerReportsAbortAsFailure(t *testing.T) {
103125
cfg := testConfigWithClaudeFixture(t)
104126
ctx, cancel := context.WithCancel(context.Background())

cmd/agentsview/usage.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -312,6 +312,7 @@ func ensureFreshData(
312312
if database.NeedsResync() {
313313
engine := sync.NewEngine(database, sync.EngineConfig{
314314
AgentDirs: appCfg.AgentDirs,
315+
SourceMachines: appCfg.SourceMachines,
315316
IncludeCwdPrefixes: appCfg.SyncIncludeCwdPrefixes,
316317
Machine: appCfg.LocalMachineName,
317318
})
@@ -336,6 +337,7 @@ func ensureFreshData(
336337

337338
engine := sync.NewEngine(database, sync.EngineConfig{
338339
AgentDirs: appCfg.AgentDirs,
340+
SourceMachines: appCfg.SourceMachines,
339341
IncludeCwdPrefixes: appCfg.SyncIncludeCwdPrefixes,
340342
Machine: appCfg.LocalMachineName,
341343
})

0 commit comments

Comments
 (0)