upstream(agentsview): duckdb watch polling owner — cherry-pick 9febfed onto main - #33
Draft
diazMelgarejo wants to merge 5 commits into
Draft
upstream(agentsview): duckdb watch polling owner — cherry-pick 9febfed onto main#33diazMelgarejo wants to merge 5 commits into
diazMelgarejo wants to merge 5 commits into
Conversation
DuckDB config resolution currently leaves a leading `~` literal in
`[duckdb].path`, so daemon-backed pushes can canonicalize the server
mirror path against the daemon working directory instead of the user's
home directory.
This expands leading-home shorthand during DuckDB config resolution,
alongside the existing defaulting and env expansion, so the daemon and
in-process callers resolve the same mirror path from the same config
value. `${HOME}` expansion, default-path fallback, and non-leading
tildes keep their current behavior. The change stays in
`internal/config` with focused resolution regression coverage.
The config snippet and daemon-vs-in-process repro came from @halms's
issue report.
Closes kenn-io#1264
---------
Co-authored-by: Marius van Niekerk <marius.v.niekerk@gmail.com>
Second PR in the artifact-sync split (follows kenn-io#1242, which froze the wire format and added the docbank-backed store). Extracted from the closed kenn-io#1239 branch with review fixes applied during extraction. ## What this adds - **SQLite publication ledger** (`internal/db`): `artifact_export_queue`, `artifact_publications`, `artifact_publication_revisions`, `artifact_checkpoint_heads`/`_floors`, plus session triggers that enqueue owned-session changes. Triggers are origin-gated — they fire only once an artifact origin exists in `pg_sync_state` — so archives that never opt in carry no queue writes. Trigger DDL lives in Go and is installed after column migrations (drops run before), so no trigger references session columns while migrations run. - **Go enqueue hooks** for child-only mutations that don't touch trigger-covered session columns: batch message writes (queue generation sampled around the batch, enqueues exactly once when the triggers didn't fire), standalone usage-event replacement, and token-coverage backfill. Queue bootstrap is an explicit call at origin creation, not a migrate-time backfill. - **Origin lifecycle** (`internal/artifact`): `EnsureOrigin`/`AdoptOrigin`/`StoredOrigin`. Creating or adopting an origin bootstraps the export queue; a failed bootstrap rolls the origin back (deleting the key when there was none before, since the gates test key existence) so a retry re-runs population. Adopting over a different established origin force-requeues every owned session with a generation bump, because prior acknowledgements belong to the old origin. - **Checkpointed export pipeline** (`internal/artifact`): claims pending queue rows, publishes content-addressed session manifests and segments into the store, records publication revisions, and advances per-origin checkpoint heads with monotone sequence reservation. Stale claims (a writer advanced the generation mid-export) roll back atomically. Incremental export is bounded by the dirty batch; an unchanged archive costs a catalog identity check only. Full export streams all bodies, then re-checks the queue up to 32 settle rounds — hitting the bound returns the accumulated result with an error meaning "made progress, run again." - **Resync carriage**: `CopySyncStateFrom` carries queue, publication, revision, and checkpoint state across a full resync and re-dirties every copied queue row — the origin gate keeps triggers silent in the rebuild's temp DB, and a parser bump changes manifest hashes anyway, so the exporter must re-verify each session (unchanged content is cheap to skip, being content-addressed). ## Scope and limitations - Export-side only. Nothing wires the pipeline to the daemon or CLI yet; that lands with folder transport and import in a later PR. There is no import/read path here. - The ledger is local SQLite state. PostgreSQL push and shared query shapes are untouched. - The wire format is unchanged: no golden churn, format stays frozen at v1. ## Where to look - `internal/db/artifact_publication.go` — queue and ledger SQL, origin-gated enqueue - `internal/db/db.go` — trigger DDL split around column migrations, bootstrap/requeue - `internal/artifact/export.go` — publish → manifest → checkpoint ordering and claim lifecycle - `internal/artifact/origin.go` — origin lifecycle and rollback semantics - `internal/db/orphaned.go` — resync carriage and the pending-flag decision Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>
Supersedes kenn-io#1167 — same feature, rebuilt as a lean series on the capability work that landed in kenn-io#1282/kenn-io#1283. The comment on kenn-io#1167 records what was cut and why. ## What this does Adds Omnigent to the supported agents. Omnigent stores every conversation in one SQLite file (`chat.db`), so this follows the same container model as Zed: sync watches one file, and the archive holds one session per conversation (`omnigent:0:<hex>`). - Scheduled syncs reparse the container when its fingerprint (mtime + hash of the db and WAL) changes. - Watcher events do a bounded, indexed scan of only the members whose `updated_at` moved, and retire disappeared members as tombstones. WAL `-shm` checkpoint noise is ignored. - Deleting a conversation (or the whole db) tombstones its sessions; nothing is destroyed. ## Notes - Supports the two schema generations that ship in released Omnigent builds, including the current binary-uuid one. The oldest shape is reported as unsupported without failing sync; the interim `session_overrides` shape is not supported. - `chat.db` co-locates transcripts with authentication secrets, so Omnigent is excluded from remote sync (capability from kenn-io#1282, nested-root enforcement from kenn-io#1283). - Validated against a chat.db produced by Omnigent's own store code at upstream head, not just fixtures (evidence in `docs/internal/session-format-sources.md`). - Known trade-offs (Zed parity): the scheduled reparse scales with container size, and edits that don't bump `updated_at` wait for the scheduled pass instead of the watcher. ## Where to look - `internal/parser/omnigent.go` — schema detection and member queries - `internal/parser/omnigent_provider.go` — discovery, fingerprinting, watcher scan, tombstones - `internal/sync/engine.go` — container scheduling and the persistent-archive audit fallback - `internal/sync/omnigent_integration_test.go` — end-to-end sync coverage Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>
Activity reports currently fragment repository activity under short-lived worktree directory names when linked worktrees are backed by a bare Git repository or their checkout has already been removed. This change derives live bare-backed repositories from the common Git directory without spawning Git, and recognizes standard hosted worktree layouts when Git metadata is unavailable. Tool-anchored manager layouts keep their explicit path precedence, while generic hosted patterns defer to a live enclosing repository so matching fixture or nested paths are not misattributed. The parser data version advances so existing source-backed sessions are reparsed through the established non-destructive full-resync flow. Archived sessions whose original source is permanently unavailable are preserved, but their previous project attribution cannot be recomputed. The main review points are the filesystem-only `core.bare` detection, the anchored-versus-generic layout precedence, and the data-version-triggered repair path. Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>
Local DuckDBPushWatch mirrored pg watch's pre-fix gap: no sync engine, no unwatched-root poller, and watcher options only wired OnCoverageDegraded. Interval pushes used runLocalSyncAuthoritative → SyncAll, which never tombstones missed deletions under deferred scopes. Align local duckdb watch with local pg watch: shared watcher options, engine-backed batch sync, startup sync, and the probe-gated poller. Add TestLocalDuckDBPushWatchGivesDeferredScopesAPollingOwner parallel to the existing pg regression at archive_write_backend_test.go:821. Daemon-delegated duckdb/pg watch remains unchanged (no local engine).
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream draft for kenn-io/agentsview#1297.
This branch is kenn-io/agentsview
main+ cherry-pick9febfedonly (re-cherry-picked as712684a4on fresh upstream clone).Target upstream PR:
kenn-io/agentsview←fix/duckdb-push-watch-polling-owner(draft)Cherry-pick source: 9febfed
Summary
Local
DuckDBPushWatchlacked the probe-gated unwatched-root poller that localPGPushWatchhas. Aligns local duckdb watch with local pg watch.Fixes kenn-io#1297