Skip to content

docs: survey the optimization candidates the campaign never looked at - #84

Open
rabbitson87 wants to merge 2 commits into
mainfrom
docs/optimization-candidates
Open

docs: survey the optimization candidates the campaign never looked at#84
rabbitson87 wants to merge 2 commits into
mainfrom
docs/optimization-candidates

Conversation

@rabbitson87

Copy link
Copy Markdown
Member

Documentation only. No code changes, no new tests, no new benchmarks.

The two-phase performance campaign finished what PLAN.md scheduled, but it only ever examined crates that already had benchmarks. This surveys the ones that do not — rpc (9,137 LOC), mempool (4,254), p2p (3,295), chain (3,010), filters (771) — and found six candidates, two of them worse than anything the campaign itself fixed.

Independent of #80 and #83; branched from main. Every file:line was re-verified against main before this was written. The cross-references out of the page are forward-looking and land with those two PRs — the findings depend on neither, and the page says so.

The two that stand out

A — gettxoutproof reads the whole chain when given no block hash. crates/rpc/src/handlers/tx.rs:175 does ctx.blocks.read().clone(), deep-copying every BlockRecord, then for each one loads the body, deserializes the whole block and computes every txid into a HashSet. At tip that is ~957,600 block loads and full deserializations to answer one call, almost all of it discarded.

The fix is already in the tree: Context.indexer (crates/rpc/src/context.rs:302) is the txindex, and the read-path campaign's resolve_transaction answers exactly "which block contains this txid". Bitcoin Core takes the same route when txindex is on.

B — the block-record log grows one entry per block, forever. crates/rpc/src/context.rs:534 pushes; nothing removes. Pruning (crates/node/src/state.rs:566) blanks block_hex but keeps the record, and the only removal is the single-entry reorg undo at crates/node/src/apply.rs:1167.

264 B per record, shown per field rather than asserted — 104 of struct plus 160 of heap for an 80-byte header stored as a hex String. That is 103.9 MiB at height 412,732 and 241.1 MiB at tip, which sizes it at 15.9% of the 0.64 GiB the attribution run left unattributed, on the one G14 budget that is at risk.

One correction worth flagging: header_hex looked dead and is not. It is returned at crates/rpc/src/handlers/chain.rs:286 and :291 and decoded at :886. So the candidate is "store 80 raw bytes, encode on read" — the read is one RPC call, the storage is every block for the life of the process.

The other four

Candidate Evidence
C mempool priority index rebuilds in O(n² log n) pareto.rs:29-42, pool.rs:419, pool.rs:620-626
D dbcache_mb reaches no storage backend zero references in crates/storage; issue #51
E load_range opens the file once per position 12.00 µs × P, measured during the read-path work
F remaining record encoding ~6 B/output, ~1.0 GiB at tip, projected

D is why the residual cannot be attributed. With no lever between config and the backends, fjall takes builder defaults and RocksDB a fixed 256 MiB block cache, so the one at-risk budget cannot be tuned and re-measured. It already cost the campaign one retracted claim.

Negative results are recorded too

So nobody re-derives them: the chain fork-point walk (crates/chain/src/tree.rs:100-118) is O(depth) rather than quadratic; Context.transactions looks like candidate B but is not populated per block by the apply path; the p2p banlist retain is a periodic expiry sweep, not per-message work.

What is measured and what is not

The ranking table marks this explicitly, because the campaign's own recorded lesson is that a plausible speedup claim is worth nothing until the harness is shaped like the workload:

  • Measured per-unit: B and E only.
  • Structural, magnitude unknown: A and C. The page says A needs its current cost timed first, so the fix has a before arm.
  • Projected from an unconverged number: F, which rests on 3.626 outputs per record (2.296 at height 183k, 4.056 at 390k).

What blocks what

A, B and C are implementable now under the refactor-set contract. D is a design question before it is an optimization. E is gated on the real G14 Electrum measurement — building it against synthetic fixtures would be the mistake this campaign already documented.

Three of the six point at the same missing measurement: G14 tip RSS on a synced mainnet-tip node with txindex and blockfilterindex. That run decides whether B and F earn their complexity, and it is the same run that decides whether the v5 codec in #83 is kept or reverted.

The two-phase campaign finished what PLAN.md scheduled, but it only ever
examined crates that already had benchmarks. Surveying the ones that do not —
`rpc` (9,137 LOC), `mempool` (4,254), `p2p` (3,295), `chain` (3,010), `filters`
(771) — found six candidates, two of them worse than anything the campaign
itself fixed.

No code changes. Each entry names what would have to be measured before it is
worth doing, because two are structural findings whose magnitude is unknown and
one is a projection from a number that has not converged.

The two that stand out:

- **`gettxoutproof` reads the whole chain** when called without a block hash
  (`crates/rpc/src/handlers/tx.rs:175`). It deep-copies every `BlockRecord`, then
  loads, deserializes and fully txid-hashes every block — ~957,600 of them at
  tip, to answer one call. The txindex is already reachable from the RPC context
  and the read-path campaign's `resolve_transaction` already answers exactly this
  question; Bitcoin Core takes the same route when txindex is on.

- **The block-record log grows one entry per block, forever**
  (`crates/rpc/src/context.rs:534`). Pruning blanks `block_hex` but keeps the
  record, and the only removal is the single-entry reorg undo. At 264 B/record —
  160 of which is an 80-byte header stored as a hex `String` — that is 103.9 MiB
  at the height the attribution run measured and 241.1 MiB at tip, which sizes it
  at 15.9% of the 0.64 GiB residual that run left unattributed. The arithmetic is
  shown per field rather than asserted.

Also records the mempool priority index rebuilding in O(n² log n), `dbcache_mb`
reaching no storage backend at all (issue #51, and the reason the residual cannot
be attributed by tuning), the deferred `load_ranges` batch read, and the two
record-encoding savings not taken.

And the negative results, so nobody re-derives them: the chain fork-point walk is
O(depth) rather than quadratic, `Context.transactions` only looks like the
block-record log but is not populated per block, and the p2p banlist `retain` is
a periodic sweep rather than per-message work.

Three of the six point at the same missing measurement — G14 tip RSS on a synced
mainnet-tip node — which is also the run that decides whether the v5 codec is
kept or reverted.
@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ac985c3c-41cf-49dc-9365-be8ed850082b


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant