Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions docs/internal/session-format-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,82 @@ Grok section and remove the explicit registry exception in the coverage test.
counts as a failure, matching the existing `exit status N` heuristic, and a
timed-out command records `timeout: true` with no `exit` key, so it is not
detected here. See #1256.
- **Change detection (SQLite layout):** every session in a root shares one
physical `opencode.db`, so the container's own size and mtime move whenever
any single session is written and cannot discriminate between sessions.
Agentsview instead builds a per-session composite from
`session.time_updated`, `project.time_updated`, `MAX(message.time_updated)`,
and `MAX(part.time_updated)` (`openCodeCompositeMtimeExpr`), and omits the
container size from the per-session fingerprint. Verified 2026-07-27 against
an isolated clone of a production container (13.5 GB, 5,981 sessions, 104k
messages, 508k parts): 432,779 of 508,400 parts (86%) carry
`time_updated != time_created`, so in-place child edits do move the signal;
437 sessions have `MAX(part.time_updated) > session.time_updated`, so the
session row alone is insufficient; and no project's `time_updated` falls
within 5s of its newest session, so folding `project` in tracks genuine
worktree/metadata changes rather than ordinary session activity. The child
scans cost ~0.6s warm on that container because `part.data` lives in SQLite
overflow pages, so scanning `(session_id, time_updated)` does not read
transcript bytes. A MAX over timestamps cannot see a deletion: on that
container 5,758 of 5,981 sessions (96%) carry a session or project timestamp
at or above every child, so removing a message or part leaves the max
untouched. The fingerprint hash therefore carries a per-session digest of the
watermark plus the child row counts, and freshness compares it
(`FingerprintHashRequiredForFreshness`). An earlier revision of this entry
claimed a revert stays detectable because it lowers the max; that is wrong for
the 96% above, and the row counts are what actually cover deletions. Known
gap: a write that leaves the watermark, the message count and the part count
all unchanged is not attributed to any session, which requires an in-place
edit that does not stamp `time_updated`. Containers whose schema lacks the
child `time_updated` columns (older OpenCode, Kilo, MiMoCode, ICodeMate) fall
back to the session-only mtime plus the container size and emit an empty
digest, preserving prior behavior. Watcher events do not pay the child scan
at all: changed-path classification lists sessions through a bounded
session-row watermark (`MAX(session.time_updated, project.time_updated)`,
`ForEachOpenCodeSessionWatermarkMeta`, ordered by session id), compares it
per session and like-for-like against the stored session/project metadata
watermark recovered from the persisted child digest
(`OpenCodeChildDigestMetadataWatermarkNS`; rows without a parseable digest
fall back to the stored composite), merged in ascending virtual-path order
against a paged stored-freshness cursor
(`ListVirtualContainerMemberFreshnessPage` through
`storedMemberFreshnessPager` and `changedWatermarkSources`), and drops
covered sessions during the stream — only the changed batch is ever
materialized, peak memory per event is one stored page plus that batch,
and the surviving sources resolve the full composite and digest through
the indexed per-session lookup. The merge trusts stored authority only
while a container capture taken before the listing still matches a
recapture afterwards; a stale capture re-lists unfiltered and leaves the
decision to the per-file gates. The comparison must be like-for-like: the
stored composite can be dominated by a newer child timestamp, and
comparing the session-row watermark against it would hide a metadata
update (title, directory, worktree rename) whose stamp lands below that
child maximum. A session or project row that advances past its own stored
metadata watermark is always a candidate, wherever other sessions'
watermarks or its own child timestamps sit. Periodic full passes and
streamed reconciliation passes over a container whose captured state still
matches the last fully verified pass also list the watermark form
(`SQLiteContainerUnchangedSinceTrust`): every member gate-skips before
fingerprinting, so the child identity scan would be archive-sized work
nothing reads; any write breaks that trust and the next pass carries the
complete digest again. Watermark-only skips additionally require the
pass's container capture to still be valid
(`sqliteContainerPassCaptureValid`) — a container that changes between
listing and the recapture check resolves full per-session digests instead,
so a concurrent child-only write cannot hide beneath an unchanged metadata
watermark. The trade is explicit: any
child-only write that leaves the session and project rows untouched —
wherever its timestamps land relative to the stored composite — is
invisible to a watcher pass and is reconciled by the next full-discovery
pass over the now-untrusted container, whose digest still catches it; on
the production container above, 96% of sessions carry a session/project
timestamp at or above every child, and actively watched sessions bypass
this entirely via the per-session composite poll. Per-event work is
bounded by the changed batch plus one O(session-count) scan of small
fixed-width rows (the session table and the paged stored-member reads);
that floor is irreducible without a watermark index, which OpenCode's
schema does not have and which is not agentsview's to add — but only the
changed batch and one stored page are ever held in memory.
- **Agentsview:** `internal/parser/opencode.go`,
`internal/parser/opencode_provider.go`, and
`internal/parser/opencode_storage_state.go`; legacy and database layouts are
Expand Down
189 changes: 182 additions & 7 deletions internal/db/sessions.go
Original file line number Diff line number Diff line change
Expand Up @@ -2231,6 +2231,141 @@ func (db *DB) GetFileInfoByPath(
return s.Int64, m.Int64, true
}

// VirtualContainerMemberFreshness is one stored virtual member's freshness
// signal: the newest stored file_mtime for its path, the minimum stored
// data version, and the newest row's fingerprint hash, mirroring
// GetFileInfoByPath, GetDataVersionByPath, and GetFileHashByPath.
type VirtualContainerMemberFreshness struct {
MTimeNS int64
DataVersion int
Hash string
}

// VirtualContainerMemberFreshnessRow pairs one virtual member path with its
// folded freshness signal.
type VirtualContainerMemberFreshnessRow struct {
Path string
VirtualContainerMemberFreshness
}

// ListVirtualContainerMemberFreshnessPage returns the freshness signal for
// stored sessions whose file_path is a virtual member of the shared container
// at containerPath ("<containerPath>#<sessionID>"), excluding source-missing
// tombstones: at most limit member paths strictly after afterPath, in
// ascending path order, and whether the container's stored membership is
// exhausted. Changed-path classification merges a streamed watermark-only
// listing against these pages, so a one-session write flows one candidate
// into the sync pipeline while peak memory stays one page — never the
// container's full membership.
//
// Two queries per page keep each path's fold complete without materializing
// the container: a DISTINCT path page rides idx_sessions_file_path ('$' is
// the ASCII successor of '#', so the half-open range covers exactly the
// "<containerPath>#" prefix), and the row fetch is bounded to that page's
// [first, last] interval, so duplicate session rows for one path can never
// split across pages. Folded in Go rather than GROUP BY: the fold needs
// MAX(file_mtime), MIN(data_version), and the hash of the newest-mtime row,
// and SQLite's bare-column-from-the-extreme-row guarantee only holds with
// exactly one min/max aggregate in the query.
func (db *DB) ListVirtualContainerMemberFreshnessPage(
ctx context.Context, containerPath, afterPath string, limit int,
) ([]VirtualContainerMemberFreshnessRow, bool, error) {
if containerPath == "" || limit <= 0 {
return nil, true, nil
}
notMissing := " AND (deletion_cause IS NULL" +
" OR deletion_cause <> '" + deletionCauseSourceMissing + "')"
lower := containerPath + "#"
lowerOp := ">="
if afterPath > lower {
lower = afterPath
lowerOp = ">"
}
pathRows, err := db.getReader().QueryContext(ctx,
"SELECT DISTINCT file_path FROM sessions"+
" WHERE file_path "+lowerOp+" ? AND file_path < ? || '$'"+
notMissing+
" ORDER BY file_path LIMIT ?",
lower, containerPath, limit,
)
if err != nil {
return nil, false, fmt.Errorf(
"listing container member paths %s: %w", containerPath, err,
)
}
defer pathRows.Close()
var paths []string
for pathRows.Next() {
var path string
if err := pathRows.Scan(&path); err != nil {
return nil, false, fmt.Errorf(
"scanning container member path %s: %w", containerPath, err,
)
}
paths = append(paths, path)
}
if err := pathRows.Err(); err != nil {
return nil, false, err
}
if len(paths) == 0 {
return nil, true, nil
}
done := len(paths) < limit

rows, err := db.getReader().QueryContext(ctx,
"SELECT file_path, file_mtime, data_version, file_hash FROM sessions"+
" WHERE file_path >= ? AND file_path <= ?"+notMissing,
paths[0], paths[len(paths)-1],
)
if err != nil {
return nil, false, fmt.Errorf(
"listing container member freshness %s: %w", containerPath, err,
)
}
defer rows.Close()
members := make(map[string]VirtualContainerMemberFreshness, len(paths))
for rows.Next() {
var path string
var mtime, version sql.NullInt64
var hash sql.NullString
if err := rows.Scan(&path, &mtime, &version, &hash); err != nil {
return nil, false, fmt.Errorf(
"scanning container member freshness %s: %w",
containerPath, err,
)
}
row := VirtualContainerMemberFreshness{
MTimeNS: mtime.Int64,
DataVersion: int(version.Int64),
Hash: hash.String,
}
member, seen := members[path]
if !seen {
members[path] = row
continue
}
if row.MTimeNS > member.MTimeNS {
member.MTimeNS = row.MTimeNS
member.Hash = row.Hash
}
if row.DataVersion < member.DataVersion {
member.DataVersion = row.DataVersion
}
members[path] = member
}
if err := rows.Err(); err != nil {
return nil, false, err
}
page := make([]VirtualContainerMemberFreshnessRow, 0, len(paths))
for _, path := range paths {
page = append(page, VirtualContainerMemberFreshnessRow{
Path: path,
VirtualContainerMemberFreshness: members[path],
})
}
return page, done, nil
}

// GetProjectByPath returns the stored project for the newest
// non-deleted session matching file_path.
func (db *DB) GetProjectByPath(path string) (project string, ok bool) {
Expand Down Expand Up @@ -2502,8 +2637,14 @@ func (db *DB) listActiveSessionSourceOwnershipScopeBatch(
for _, scope := range scopes {
root := scope.Path
likeRoot := sqliteLikeEscape(root)
// A drive root or share root already ends in a separator, so the
// child prefix must not add a second one: stored children carry one.
childPrefix := likeRoot + string(filepath.Separator) + "%"
if strings.HasSuffix(root, string(filepath.Separator)) {
childPrefix = likeRoot + "%"
}
rootClause := `(b.file_path = ? OR b.file_path LIKE ? ESCAPE '!')`
args = append(args, root, likeRoot+string(filepath.Separator)+"%")
args = append(args, root, childPrefix)
if scope.IncludeVirtualMembers {
// Mirror storedSourcePathHintInRoot: a virtual member is the
// container plus '#' and a nonempty single segment. Nested stored
Expand Down Expand Up @@ -2979,17 +3120,51 @@ func storedSourcePathHintInAnyRoot(
return false
}

// StoredSourcePathHintScopesContain reports whether one stored or discovered
// source path lies inside any of the bounded scopes, using the same
// platform-aware containment the SQLite queries in this file re-check. It is
// the single Go-side authority for path-to-scope membership: the SQL LIKE
// prefilters stay a superset of this predicate for ASCII paths, and every
// caller that pages or admits by scope must apply it rather than comparing
// prefixes itself.
func StoredSourcePathHintScopesContain(
path string, scopes []StoredSourcePathHintScope,
) bool {
return storedSourcePathHintInAnyRoot(path, scopes)
}

func storedSourcePathHintInRoot(path string, scope StoredSourcePathHintScope) bool {
path = cleanStoredSourcePathHint(path)
root := cleanStoredSourcePathHint(scope.Path)
if path == root || strings.HasPrefix(path, root+string(filepath.Separator)) {
if storedSourcePathSameOrDescendant(path, root) {
return true
}
suffix, ok := strings.CutPrefix(path, root+"#")
return ok &&
scope.IncludeVirtualMembers &&
suffix != "" &&
!strings.ContainsAny(suffix, `/\`)
if !scope.IncludeVirtualMembers || len(path) < len(root)+2 ||
path[len(root)] != '#' {
return false
}
// A virtual member is the container plus '#' and a nonempty single
// segment. The container prefix folds with the same platform semantics as
// the directory branch; the member segment rule stays byte-exact.
if rel, err := filepath.Rel(root, path[:len(root)]); err != nil || rel != "." {
return false
}
member := path[len(root)+1:]
return !strings.ContainsAny(member, `/\`)
}

// storedSourcePathSameOrDescendant compares with filepath.Rel semantics so
// containment matches platform path equality: case-folded per element on
// Windows, byte-exact on Unix. This keeps the Go predicate aligned with the
// ASCII-case-insensitive SQL LIKE prefilter on Windows while staying exact on
// case-sensitive filesystems.
func storedSourcePathSameOrDescendant(path, root string) bool {
rel, err := filepath.Rel(root, path)
if err != nil {
return false
}
return rel == "." || rel != ".." &&
!strings.HasPrefix(rel, ".."+string(filepath.Separator))
}

func sqliteLikeEscape(value string) string {
Expand Down
Loading