Skip to content

Commit 05204d4

Browse files
committed
fix(artifact): bound folder transport replay
Folder exchanges must not rescan and rehash every immutable object while holding the daemon sync lock, because archive history grows independently of current work. Bind durable continuation state to a unique target namespace, consume an immutable publication journal within object and byte budgets, and resume authoritative closure publication by checkpoint generation. Unchanged heads now avoid closure traversal, while interrupted journal and quarantine transitions remain replayable and fail closed.
1 parent 5d6463a commit 05204d4

10 files changed

Lines changed: 1273 additions & 115 deletions

docs/artifact-sync.md

Lines changed: 23 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
1-
---
2-
title: Artifact Folder Sync
3-
description: Exchange normalized sessions between AgentsView archives through a trusted folder
4-
---
1+
______________________________________________________________________
2+
3+
## title: Artifact Folder Sync description: Exchange normalized sessions between AgentsView archives through a trusted folder
54

65
Artifact folder sync exchanges normalized AgentsView sessions between machines
76
through a folder that both machines can access. The folder can be a mounted NAS,
@@ -17,16 +16,19 @@ folder, or change ordinary provider sync behavior.
1716

1817
!!! warning
1918

20-
The target contains normalized session messages and related metadata. It can
21-
contain sensitive prompts, responses, tool activity, and usage data. Use only a
22-
folder whose storage and access controls you trust.
19+
```
20+
The target contains normalized session messages and related metadata. It can
21+
contain sensitive prompts, responses, tool activity, and usage data. Use only a
22+
folder whose storage and access controls you trust.
23+
```
2324

2425
## First use
2526

2627
The target must either not exist or be an empty directory. On first use,
2728
AgentsView creates it and adds an `.agentsview-artifacts.json` namespace marker.
28-
Later runs refuse an unmarked nonempty directory rather than adopting unrelated
29-
files.
29+
The marker includes a randomly generated namespace identity used to bind local
30+
continuation state to this exact target. Later runs refuse an unmarked nonempty
31+
directory rather than adopting unrelated files.
3032

3133
Run the command once on each participating machine:
3234

@@ -40,13 +42,16 @@ agentsview sync --target /mnt/team-agentsview
4042

4143
The path can differ between machines. Each archive pulls verified immutable
4244
objects already in the folder, publishes its own objects, and imports supported
43-
peer checkpoints into its local SQLite archive. Repeating the command is safe:
44-
already accepted content is verified and skipped, while changed sessions publish
45-
a new revision.
45+
peer checkpoints into its local SQLite archive. Publication also appends an
46+
immutable change-journal entry. Each machine retains a target-bound journal
47+
cursor, so repeating the command processes new entries instead of rescanning and
48+
rehashing the folder's complete history. Changed sessions publish a new
49+
revision.
4650

47-
The command performs a bounded amount of work so concurrent provider writes
48-
cannot keep it running forever. If the summary says that artifact work remains,
49-
run the same command again:
51+
The command enforces object-count and decoded-byte budgets for both inbound and
52+
outbound transport work. Continuation cursors are durable, so later runs resume
53+
after the last accepted object rather than restarting at the first object. If
54+
the summary says that artifact work remains, run the same command again:
5055

5156
```text
5257
Artifact work remains; run the sync command again.
@@ -96,6 +101,9 @@ restored from backup, keep only one active writer for that restored origin.
96101
machine retains imported sessions in its own SQLite archive.
97102
- Objects are content-addressed and immutable. Conflicting content under an
98103
existing identity fails closed.
104+
- Journal entries are immutable and sequence numbered. AgentsView advances a
105+
local cursor only after the referenced object has been read, verified, and
106+
accepted, making interruption and retry safe.
99107
- Invalid complete objects are quarantined so one corrupt peer artifact does not
100108
permanently block unrelated origins.
101109
- Folder operations are serialized between cooperating AgentsView processes. Do

internal/artifact/sync.go

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,10 @@ func SyncWithRepository(
9595
forbidden = append(forbidden, repository.rootPath)
9696
transport, err := OpenFolderTransport(
9797
opts.Target,
98-
FolderTransportOptions{ForbiddenRoots: forbidden},
98+
FolderTransportOptions{
99+
ForbiddenRoots: forbidden,
100+
StateStore: databaseFolderTransportState{database: database},
101+
},
99102
)
100103
if err != nil {
101104
return SyncResult{}, err
@@ -154,10 +157,29 @@ func SyncWithRepository(
154157
if err != nil {
155158
return result, err
156159
}
157-
result.More = exportMore || importMore
160+
result.More = exportMore || exchanged.More || importMore
158161
return result, nil
159162
}
160163

164+
type databaseFolderTransportState struct {
165+
database *db.DB
166+
}
167+
168+
func (s databaseFolderTransportState) LoadFolderTransportState(
169+
_ context.Context,
170+
namespaceID string,
171+
) (string, error) {
172+
return s.database.GetSyncState("artifact_transport_" + namespaceID)
173+
}
174+
175+
func (s databaseFolderTransportState) SaveFolderTransportState(
176+
_ context.Context,
177+
namespaceID string,
178+
value string,
179+
) error {
180+
return s.database.SetSyncState("artifact_transport_"+namespaceID, value)
181+
}
182+
161183
func validateArtifactSyncOptions(opts SyncOptions) error {
162184
if strings.TrimSpace(opts.Target) == "" {
163185
return fmt.Errorf(

internal/artifact/sync_test.go

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"bytes"
55
"context"
66
"fmt"
7+
"os"
78
"path/filepath"
89
"testing"
910
"time"
@@ -69,6 +70,7 @@ func TestArtifactSyncTwoNodeFolderRoundTripAndReplay(t *testing.T) {
6970
require.NoError(t, err)
7071
require.Len(t, messages, 2)
7172
assert.Equal(t, "world", messages[1].Content)
73+
journalSequenceBeforeReplay := readTestFolderJournalSequence(t, target)
7274

7375
replay, err := SyncWithRepository(
7476
t.Context(),
@@ -80,6 +82,12 @@ func TestArtifactSyncTwoNodeFolderRoundTripAndReplay(t *testing.T) {
8082
assert.Zero(t, replay.ImportedSessions)
8183
assert.Zero(t, replay.ImportedMessages)
8284
assert.False(t, replay.More)
85+
assert.Equal(
86+
t,
87+
journalSequenceBeforeReplay,
88+
readTestFolderJournalSequence(t, target),
89+
"an unchanged authoritative head must not replay its closure",
90+
)
8391

8492
require.NoError(t, databaseA.ReplaceSessionMessages("one", []db.Message{
8593
{
@@ -119,6 +127,16 @@ func TestArtifactSyncTwoNodeFolderRoundTripAndReplay(t *testing.T) {
119127
assert.Equal(t, "updated response", updatedMessages[1].Content)
120128
}
121129

130+
func readTestFolderJournalSequence(t *testing.T, target string) int64 {
131+
t.Helper()
132+
root, err := os.OpenRoot(filepath.Join(target, folderJournalDirectory))
133+
require.NoError(t, err)
134+
head, err := readFolderJournalHead(root)
135+
require.NoError(t, err)
136+
require.NoError(t, root.Close())
137+
return head.Sequence
138+
}
139+
122140
func TestArtifactSyncDoesNotIngestOrRepublishSpoofedLocalOrigin(
123141
t *testing.T,
124142
) {

internal/artifact/transport.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,23 @@ type Transport interface {
2222
type ExchangeResult struct {
2323
Received int
2424
Published int
25+
More bool
2526
}
2627

2728
// FolderTransportOptions defines roots that must remain disjoint from the
2829
// external artifact target.
2930
type FolderTransportOptions struct {
3031
ForbiddenRoots []string
32+
MaxObjects int
33+
MaxBytes int64
34+
StateStore FolderTransportStateStore
35+
}
36+
37+
// FolderTransportStateStore persists target-bound continuation state between
38+
// bounded exchanges. Implementations must treat namespaceID as an opaque key.
39+
type FolderTransportStateStore interface {
40+
LoadFolderTransportState(context.Context, string) (string, error)
41+
SaveFolderTransportState(context.Context, string, string) error
3142
}
3243

3344
type transportChangeRecorder interface {

0 commit comments

Comments
 (0)