Add per-run seq column to strategy_data archive tables - #89
Conversation
Archive rows are delivered at-most-once over a bounded queue, so a lost row and an event that never happened are currently indistinguishable. Add `seq`, a monotonic per-(controller_id, run_id) counter shared across every strategy_data table, so gap detection becomes a query over holes in the sequence. The column ships on both the CREATE TABLE definitions and as idempotent ALTER ... ADD COLUMN IF NOT EXISTS statements, so existing deployments pick it up when the forwarder applies the schema at startup. Document the delivery contract in the schema header, including the ordering constraint: inserts use JSONEachRow, so a producer emitting a column ClickHouse does not have fails the whole per-table batch. The forwarder must be deployed before the producer.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
1 participant
Why
Archive rows reach ClickHouse at-most-once over a bounded in-memory queue: the producer drops the oldest row when the queue is full, and discards a batch after two failed POST attempts. Nothing on the row makes that loss observable, so today a missing row and an event that never happened are indistinguishable.
That is not just a telemetry gap. It makes the order-identity acceptance queries unprovable rather than merely unverified — "every ladder level has a placed-or-rejection row" cannot be answered when a missing row has two possible explanations.
What
Adds
seqto all fivestrategy_datatables: a monotonic counter scoped to one(controller_id, run_id)and shared across every table, so gap detection becomes a UNION over the tables filtered byrun_id, looking for holes.The column ships both in the
CREATE TABLEbodies and as idempotentALTER TABLE ... ADD COLUMN IF NOT EXISTSstatements, following the existingsource_cursorprecedent, so already-created tables pick it up when the forwarder applies the schema at startup. Existing rows keepseq = 0; only runs starting after the producer ships allocate real values.The schema header now documents the delivery contract: what
seqproves, what it does not, and the deploy ordering below.Deploy ordering — please read
Inserts use
JSONEachRow, so a row carrying a column ClickHouse does not have fails the whole per-table batch; the forwarder returns non-2xx and the producer drops it. This must be deployed before the producer change (usherlabs/fiet-maker, "Detect archive row loss with per-run sequence numbers"). No manual SQL step is needed —ensureArchiveSchemaapplies this file at forwarder startup.A
seqhole proves an allocated row was lost. The converse does not hold: paths that skip emission entirely never allocate aseq, so their absence leaves no hole.Verification
bun test test/archive-forwarder-contract.test.ts test/archive-forwarder.test.ts— 15 passed, 0 failed.The shared
test/fixtures/archive_forwarder_envelope.jsonis byte-identical to its counterpart in the producer repo; both are updated together in this pair of changes.