pg: add --from-now to start a new push target at the join point - #1267
pg: add --from-now to start a new push target at the join point#1267Technophobe01 wants to merge 4 commits into
Conversation
A target's first push has no watermark, so it backfills the entire local archive. Pushing into a database shared with other people therefore discloses every session on the machine, including unrelated local work, and there was no way to ask for "only what happens from here on". Add SyncOptions.PushFromNow, exposed as `pg push --from-now`, which seeds the watermark with the push cutoff when a target has no history yet. It is applied after the existing reset checks rather than before: those treat a watermark with no matching target fingerprint or PG-side push marker as corrupt local state and clear it, so a watermark seeded earlier (or from outside the process) is wiped and the push falls back to a full backfill. It is also ignored for an explicitly requested full push and for a target that already has history, so it can only bound a first push, never open a gap in an established one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
roborev: Combined Review (
|
Fixes from the review of the previous commit. The flag was silently dropped on daemon-delegated pushes: when the local daemon owns the archive, `pg push` posts to /api/v1/push/pg, and neither the request nor the handler carried it, so the whole archive was backfilled anyway. It now travels in the request and maps into SyncOptions. Whether --full was requested cannot be decided inside Push: callers pass `cfg.Full || didResync`, so any automatic resync looked like explicit intent and disabled the flag. The precedence is resolved at the CLI instead, where the user's own flag is known. The boundary scoped session selection only, while other phases stayed archive-wide, so pre-boundary content was uploaded regardless. A bounded push now refuses when a vector source is attached, because embeddings carry raw text and cannot be bounded, and skips cursor usage events for the same reason a filtered push already skips them: the rows are global and unattributed. Freshness is now proven rather than inferred. The reset paths clear the watermark, so an empty watermark alone could not distinguish a genuinely new target from an established one whose marker was lost or whose first push failed part-way, and re-seeding there would permanently skip the history those resets exist to restore. Boundary state, marker presence, and whether a reset ran on this pass are all required. Also documents the flag and its interactions in the command reference.
roborev: Combined Review (
|
Daemon delegation sent --full and --from-now unchanged, so the precedence resolved on the local path did not apply when the daemon owns the archive: `pg push --full --from-now` could backfill locally but bound remotely. The delegated request now carries the resolved value. Freshness is judged on the state as first read, before the recovery paths can clear the watermark and boundary state, so a target whose PG marker was lost no longer looks brand new and cannot be re-bounded. The incompatible vector configuration is refused before connecting rather than after, so watch mode does not retry a combination that can never succeed having already synced locally and opened a stream. The flag help now states the requirement. Known limitation, left for maintainer judgement: the boundary is per-push, so phases that are archive-wide by nature (cursor usage events on later pushes, project-identity publication) are only skipped on the bounded push itself. Making that permanent needs the boundary persisted per target, and the right treatment of identity publication is a product call: skipping it entirely leaves a bounded target without project identities, while scoping observations and snapshots to admitted sessions preserves them but is a larger change.
The CLI refusal added last round was itself a regression. Being flag-based it rejected targets with vector pushing disabled and established targets where --from-now is a documented no-op, and with --all one vector-enabled target aborted every other one. It also sat only in the one-shot entry point, so watch mode never reached it. Removed: the authoritative condition (the boundary actually applying with a vector source attached) is known inside Push, which already refuses there. --full precedence over --from-now is now settled once, at both command entry points, rather than at each backend. The watch backend clears Full after its startup cycle, so a retry of `--full --from-now` could previously resurrect the boundary and permanently skip history. The push handler no longer trusts the caller to have resolved it either. Known limitations, unchanged and left for maintainer judgement: the boundary is per-push, so project-identity publication and cursor usage events on later pushes are still archive-wide. Both need the boundary persisted per target, and whether identity publication should be skipped (leaving a bounded target with no project identities) or scoped to admitted sessions is a product call.
roborev: Combined Review (
|
1 participant
*** Alternate Solution Found ***
Closing this: it is the wrong approach.