Send the faded order hashes that caused a circuit breaker block in the blocking notification - #463
Draft
claude[bot] wants to merge 6 commits into
Draft
Send the faded order hashes that caused a circuit breaker block in the blocking notification#463claude[bot] wants to merge 6 commits into
claude[bot] wants to merge 6 commits into
Conversation
…otification Fillers blocked by the circuit breaker receive a notification containing only blockUntilTimestamp, so they cannot tell which order triggered it. Thread the QuoteRequest into WebhookQuoter.notifyBlock and include the order's requestId (and quoteId when present, e.g. hard quotes) in the notification payload. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
…tion
Per review feedback, the block notification now identifies the triggering
order by its on-chain order hash: HardQuoteRequest.toQuoteRequest() threads
order.hash() onto the QuoteRequest (internal only; excluded from wire
payloads), and notifyBlock sends { orderHash, quoteId } for hard quotes.
Soft quotes have no signed order at quote time, so they fall back to
requestId to keep an identifier in the notification.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
…e notification
Per review, the block notification should name the order(s) whose fades
caused the block, not the order currently being quoted. The fade-rate-v2
cron now selects orderHash from the latestRfqsV2 view (verified against
posted_orders.yaml in data-eng-workflows), collects the hashes of each
filler's new faded orders, and persists them on the block entry in Dynamo
(additive fadedOrderHashes list attribute; capped, deduped, carried
forward while the block is active, cleared when it expires). The circuit
breaker provider surfaces them on disabled endpoints and notifyBlock sends
{ blockUntilTimestamp, orderHashes, quoteId? }. requestId is gone from the
payload, and legacy block entries without stored hashes simply omit
orderHashes. The previous threading of the current request's order hash
through QuoteRequest is reverted as no longer needed.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
…rate) Re-applies the fadedOrderHashes threading onto the rewritten cron: getFillersFadeStats now also collects the faded order hashes per cohort (post-block window and during-block), and calculateNewTimestamps persists the hashes of whichever cohort(s) tripped or extended the block. Repository and provider changes carried over onto the renamed row fields (lastExaminedTimestamp/fadeWindowStart) and number-typed attributes; the notification payload behavior is unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
Keeps the orderHash selection in the V2 fade query; takes upstream's removal of the legacy V1 FadesRepository and the un-exported V2_FADE_RATE_SQL (no remaining importers). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
…tchlist Re-applies the fadedOrderHashes threading onto the reworked cron: getFillersFadeStats still collects the faded order hashes per cohort, and calculateNewTimestamps persists them through the new consecutiveCleanRuns branches (extend/new-block restart the streak and set hashes; blocked-clean carries them; the under-threshold branch omits the attribute so the full-item put clears it). Repository gains the list attribute alongside consecutiveCleanRuns. Notification payload behavior is unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018oU7bRX2dPnye7vC54t3Uz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Christian Angelopoulos · Slack thread
Before / After
Before: when the circuit breaker blocks a filler, the blocking notification POSTed to their endpoint contains only
{ blockUntilTimestamp }— fillers cannot tell which order(s) triggered the block.After: the notification names the faded order(s) that caused the block — the hashes of the posted orders the filler faded, which is what fired the circuit breaker in the first place. This works for both soft- and hard-quote requests, since the cause of a block is always a posted order:
{ "blockUntilTimestamp": 1783970000, "orderHashes": ["0x9f2b...c41a", "0x77aa...01ce"], "quoteId": "b71e2a10-..." // current request's quoteId, when it carries one (hard quotes) }requestIdis no longer sent. Block entries written before this change have no stored hashes; for those,orderHashesis simply omitted during the transition (entries refresh on every cron run, so this window is short).How
The fade-rate-v2 cron is the block writer: it queries the
latestRfqsV2Redshift view for each filler's recent orders and whether each faded, computes per-filler Laplace-smoothed fade rates, and writes the block entry per filler to the Dynamo timestamps table. This change threads the faded orders' hashes along that same path (kept current with #454's rate-based trigger, #465/#473's cleanups, and #482's clean-run decay streak via merges):lib/repositories/fades-repository.ts: thelatestRfqsV2view andV2_FADE_RATE_SQLnow also selectorderHash(column verified againstposted_orders.yamlindata-eng-workflows, where it is also a dedupe key — present on every row);V2FadesRowTypegainsorderHash.lib/cron/fade-rate-v2.ts:getFillersFadeStats()now also collects the faded orders' hashes per cohort (windowFadedOrderHashesfor the post-block rate window,duringBlockFadedOrderHashesfor the in-flight-during-block cohort).calculateNewTimestamps()persists on the block entry the hashes of whichever cohort(s) tripped or extended the block: set on a new block, appended (deduped, capped atMAX_FADED_ORDER_HASHES = 50, most recent kept) on extension, carried forward while the block is active, and cleared when the filler drops back under threshold.lib/repositories/base.ts+lib/repositories/timestamp-repository.ts+lib/constants.ts: additive optionalfadedOrderHasheslist attribute on the timestamp row — written only when present, read back asundefinedon legacy rows, no changes to existing attributes.lib/providers/circuit-breaker/{index,dynamo,mock}.ts:EndpointStatuses.disabledentries surfacefadedOrderHashes.lib/quoters/WebhookQuoter.ts:notifyBlock()sendsorderHashesfrom the block entry (omitted when the entry has none) plus the current request'squoteIdwhen present;requestIdremoved.lib/entities/{QuoteRequest,HardQuoteRequest}.ts: unchanged from main — the earlier threading of the current request's order hash is no longer needed.Testing
jestunit suite on the merged result (baseacce427): 31 suites passed, 300 tests passed. New coverage: per-cohort faded-hash collection in the cron, block/extend/carry-forward/clear/cap persistence semantics, Dynamo round-trip incl. legacy rows without the attribute, provider propagation, and exact notification payloads incl. the legacy-entry case.tscbuild clean; touched files passprettier --checkandeslintwith no errors and no new warning patterns.Context: reported by a filler (Thomas Retraint) — blocked fillers receive the notification but cannot identify the order that triggered the circuit breaker.