Skip to content

fix(sync): keep identical partial fills from broker sync - #54

Merged
ChristopherDownie merged 2 commits into
mainfrom
fix/sync-identical-fills
Oct 26, 2026
Merged

ChristopherDownie merged 2 commits into
mainfrom
fix/sync-identical-fills

Conversation

@ChristopherDownie

Copy link
Copy Markdown
Contributor

Fixes broker sync dropping genuine partial fills when one order fills in identical pieces (#50, #52).

Cause

Executions are deduplicated by a content hash of symbol, side, quantity, price and time. broker-sdk trades carry no fill ID, so when an order fills in several pieces with the same size and price at the same timestamp, every piece after the first hashed the same and was dropped as a duplicate. Volume, fees and P&L were under-counted, and a position closed by those pieces could stay open. #50 is this case: a Hyperliquid short closed by identical partial fills stayed partly open, so it merged into the next short on the same coin and the closed trade never appeared.

Fix

  • brokerTradesToExecutions numbers the later copies of an identical fill within a batch (occurrence on ImportedExecution), and executionHash includes that number. The first copy keeps its existing hash, so fills synced before this change still deduplicate and nothing is imported twice. The next sync adds the missing copies that are still in the broker's history. Copies are counted rather than indexed, so a reordered batch, or a later batch that sees fewer copies, maps onto the same hashes. IBKR Flex XML uploads share this mapping and stay deduplicated against sync.
  • IBKR sync now reads only execution rows. broker-sdk's Flex parser reads every <Trade> row as a fill, including order, closed-lot and summary rows that repeat the executions they summarize. File uploads already strip those rows; sync did not, so a query with order-level rows stored each multi-execution order as an extra fill, and with numbering the order row of a single-execution order would also be kept. Sync now passes broker-sdk a fetch (its documented option) that removes non-execution rows with the same filter uploads use.

The numbering relies on each SDK Trade being one fill, as the type documents; broker-sdk's own diffSnapshots counts identical trades the same way. I checked every adapter in broker-sdk 0.5.1 for a path that could emit one fill twice in a snapshot. Ten return no trades, and alpaca, etrade, hyperliquid, kucoin, public, questrade, schwab, tastytrade, topstep and tradier have none. IBKR Flex was the exception, handled above. robinhood-crypto appends cursor pages without deduplicating by order ID, so a cursor that shifted mid-paging could repeat an order; that is left as is.

Worth raising in broker-sdk: passing each fill's native ID on Trade (Hyperliquid tid, IBKR ibExecID), so the journal can key on it instead of counting copies, and skipping non-execution <Trade> rows in parseFlexStatement.

Test

  • apps/web/tests/sync-identical-fills.test.ts runs the Hyperliquid account sync #50 sequence through syncAccount and broker-sdk's real Hyperliquid adapter, with fetch stubbed. The first short shows as closed and the second as open; a re-sync adds nothing; an account synced before this fix keeps its fills and gains only the missing copy; a reordered or shrinking batch adds nothing; and the copy number is validated. Before the fix, the sequence stores 3 of its 4 fills and shows one open short.
  • A new case in ibkr-sync-timezone.test.ts syncs two identical executions plus repeated order rows and gets 3 fills and one closed trade. Without the fetch filter it stores 5.

Checks run locally

pnpm format:check, pnpm typecheck, pnpm test (66 files, 625 tests), node scripts/check-licenses.mjs, pnpm -r --filter "./packages/*" build and pnpm build all pass.

Fixes #50
Fixes #52

Broker sync dedupes executions by a content hash of symbol, side,
quantity, price and time. broker-sdk trades carry no fill ID, so one
order that fills in identical pieces collapsed into a single execution:
volume, fees and P&L were under-counted, and a position closed by those
pieces could stay open (#50: a closed Hyperliquid short merged into the
next short on the same coin).

brokerTradesToExecutions now numbers the later copies of an identical
fill within a batch, and the content hash includes that number. The
first copy keeps its existing hash, so fills synced before this change
still deduplicate and nothing is imported twice; the next sync adds the
missing copies that are still in the broker's history. Copies are
counted rather than indexed, so a reordered batch, or a later batch that
sees fewer copies, maps onto the same hashes. IBKR Flex XML uploads share
this mapping and stay deduplicated against sync.

Fixes #50
Fixes #52
broker-sdk's Flex parser reads every <Trade> row as a fill, including
order, closed-lot and summary rows that repeat the executions they
summarize. File uploads already strip those rows before parsing; sync
did not, so a query that includes order-level rows stored each
multi-execution order as an extra fill. With identical fills now
numbered, the order row of a single-execution order would also be kept
next to its execution.

Sync now passes broker-sdk a fetch that removes non-execution <Trade>
rows from the statement, using the same filter as uploads, so both paths
journal the same executions.
@ChristopherDownie
ChristopherDownie merged commit f4352b1 into main Oct 26, 2026
1 check passed
@ChristopherDownie
ChristopherDownie deleted the fix/sync-identical-fills branch October 28, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broker sync dedup can merge genuinely distinct fills (under-counts volume and P&L) Hyperliquid account sync

1 participant