Archive durable fill and transfer execution facts - #61
Conversation
Add broker_execution.transfer_events and broker_execution.fill_events end to end: row builders, emit sites, forwarder support, and schema. Transfers: withdrawals (including the venue fee ccxt already returns, previously discarded, and failed attempts with error_summary), observed deposits, and sub<->master internal transfers. Fills: a FillArchivePoller drives ccxt trade-history per (account, symbol) seen through order activity, with an in-memory since-cursor. Venue createOrder/GetOrderDetails responses carry no per-trade data in production, so trade-history polling is the only source of per-fill fees. Restart re-scans a bounded lookback; duplicates resolve at read time (plain MergeTree, per the consumer contract). Column shapes follow the fiet-maker CEX execution archive contract; divergences are additive only (transfer fee columns, trade_history_fill event kind, internal_transfer lifecycle values). Archive self-health: writer now emits cex_archive_rows_flushed_total and a last-flush-success gauge; the forwarder names rejected tables in its warning and response body. Unknown tables 400-reject the whole batch and requeue, so broker and forwarder must release together. No new environment variables (poller tuning is constants), keeping the Gramine manifest allowlist unchanged.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (22)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Summary
Adds durable CEX execution facts for the two grains that today require repeated, truncation-prone venue-history scans:
broker_execution.transfer_eventsandbroker_execution.fill_events, end to end (row builders, emit sites, forwarder support, schema).transfer_events
status="failed"witherror_summary).event_kind="internal_transfer".fill_events
createOrder/GetOrderDetailsresponses carry no per-trade data and no fees (verified against live archive rows: 0 of ~520k detail rows havetrades[]or fee fields), so per-fill truth requires the trade-history endpoint.FillArchivePoller(constructed only when archiving is enabled; mirrors the deposit-reconciler lifecycle) polls ccxt trade history per (account, symbol) seen through order activity, with an in-memory since-cursor. Restart re-scans a bounded lookback; duplicates resolve at read time (plain MergeTree, per the consumer contract). Symbols poll sequentially to stagger venue calls; per-symbol failures are isolated. Enclave-safe (node http/https only).Contract alignment
Column shapes follow fiet-maker's
docs/CEX_EXECUTION_ARCHIVE_CONTRACT.md(and its sandbox proof harness); a test asserts a fully-populated row covers every contract column for both tables. Deliberate divergences are additive only:transfer_eventsgainsfee_amount/fee_currency(contract lacks them; ccxt provides them).fill_events.event_kind=trade_history_fill(the contract'screate_order_fillgrain would yield zero rows in production, per the finding above);fill_indexis the trade's position within the poll batch.internal_transfer/submit_internal_transfervalues in existing enum-style columns; withdrawal failures use the contract'serror_summary.These should be reflected back into the contract doc/fixture on the fiet-maker side.
Archive self-health
cex_archive_rows_flushed_totaland acex_archive_last_flush_successgauge (the enqueued/shed/failure counters existed but flushed/last-success did not).Verification
bun test: 405 pass / 0 fail;tsc --noEmitclean; biome clean. New tests cover builder shapes against the consumer contract, fee-as-string precision preference, forwarder acceptance of the new tables + rejected-table naming, order-activity tracking, cursor advancement, and poller failure isolation.Rollout