Skip to content

Commit ade65e7

Browse files
rabbitson87claude
andcommitted
perf(mempool): make insertion cost independent of mempool size
Replacing the priority index was not enough. `Mempool::insert_entry` called `recompute_all_metadata`, which walked every entry, and then `total_vsize()`, which folded every entry again. With the index already fixed, insertion still measured an exponent of 2.17 -- 1.057 s to fill 3,200 transactions. Three changes remove what was left. The metadata refresh is incremental. Linking one transaction into the spend graph changes package totals for its transitive ancestors, itself, and its transitive descendants, and for nothing else: an entry outside that set gains no new ancestor because it is not a descendant of the seed, and gains no new descendant because every new path runs through the seed, which would have put it among the seed's ancestors. `insert_entry` and `remove_entries` now recompute exactly that closure, whose size is bounded by the ancestor and descendant policy limits rather than by the pool. The closure is taken after the entry enters the spend indexes on an insertion, because a transaction can arrive after something that already spends its outputs; and before the removal on a removal, because a removed entry's ancestors cannot be walked once it is gone. `total_vsize` and `aggregate_fees` are running sums. Both were folds over the whole pool, and `insert_entry` consults `total_vsize()` on every acceptance, so that fold alone made insertion quadratic. Measured end to end, against the same fixture as the previous commit: 200 tx 2.572 ms -> 373.0 us 6.9x 800 tx 51.27 ms -> 1.760 ms 29.1x 3,200 tx 1.057 s -> 7.079 ms 149.3x 12,800 tx unmeasurable -> 37.87 ms 51,200 tx unmeasurable -> 211.5 ms Per transaction 1.87 us -> 4.13 us across 256x the entries; measured exponent 1.24 over the final leg, which is n log n for the fill. `prioritise` had a defect this fixes on the way. It applied its fee delta to each descendant's `ancestor_fee` by hand and never reindexed those descendants, so a descendant kept the priority key it had before its ancestor was bumped -- defeating `prioritisetransaction` for exactly the packages it aims at. The three delta loops are now one call to the same refresh an insertion does. `recompute_all_metadata` is retained under `cfg(test)` as the oracle. Nothing in the pool reaches it, so production cannot drift away from it. Nine mutations, all killed on their named tests. Two survived the first pass: `aggregate_fees` guards itself with a debug_assert, but a guard only fires when something calls it, and no test called it after a removal or a fee bump. Both paths could lose their bookkeeping silently. Covered now, and compared against an independent fold so the check survives a release build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 5a78f96 commit ade65e7

3 files changed

Lines changed: 509 additions & 57 deletions

File tree

crates/mempool/benches/pareto.rs

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,13 @@ use criterion::{Criterion, criterion_group, criterion_main};
3636
/// there in reasonable time, which is itself the finding.
3737
const FILL_SIZES: [usize; 4] = [1_000, 4_000, 16_000, 50_000];
3838

39-
/// End-to-end sizes, an order of magnitude smaller: `insert_entry` rebuilds the
40-
/// entire index per transaction, so its cost is the index fill cost multiplied
41-
/// by the number of transactions.
42-
const POOL_SIZES: [usize; 3] = [200, 800, 3_200];
39+
/// End-to-end sizes.
40+
///
41+
/// The first three are the sizes the quadratic `insert_entry` could be measured
42+
/// at before the metadata refresh was made incremental — 3,200 transactions took
43+
/// a second — and are kept so the two revisions of this page compare directly.
44+
/// The last two are only reachable now, and are what pins the exponent.
45+
const POOL_SIZES: [usize; 5] = [200, 800, 3_200, 12_800, 51_200];
4346

4447
fn spread_fee(seed: u64) -> u64 {
4548
// Not monotonic in the seed: an index fed entries already in priority order

0 commit comments

Comments
 (0)