Skip to content

Commit dcb1929

Browse files
rabbitson87claude
andcommitted
test(mempool): stop choosing test subjects out of a HashMap
`Mempool::by_txid` is a `HashMap`, so its iteration order is randomized per process. Two of the tests added with the incremental metadata refresh picked their subject out of it — `victims.get(3)` and `by_txid.keys().next()` — and so removed or bumped a different transaction on every run. That made the mutation audit unsound rather than merely noisy. Rebuilding the "a removal takes its closure after the removal" mutant and running the test binary 60 times put the removal equivalence test at 36 red out of 60: the row recorded as a kill was a coin flip that had landed the right way. Every subject is now addressed by entry id, which is the slab index and so follows insertion order, and the removal test removes each of the six fixture entries in turn instead of one. Re-audited at 40 executions per mutant, every removal-side row is 40/40 red and the baseline 0/40. The exhaustive sweep also reaches a shape no single chosen victim could: removing `leaf` takes the fan-in `joined` with it while `sibling` — `joined`s other parent — survives and has to drop it from its descendant totals. Adds `eviction_during_insertion_leaves_metadata_a_rebuild_agrees_with` for the other gap the pass found. `enforce_size_limit` is a `remove_entries` caller reached from inside `insert_entry`, removing packages the caller never named, and nothing drove the refresh through it. It now kills both the forgets-ancestors and the skips-the-reindex mutations. No production code changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent ade65e7 commit dcb1929

2 files changed

Lines changed: 128 additions & 24 deletions

File tree

crates/mempool/src/pool.rs

Lines changed: 100 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1797,29 +1797,104 @@ mod tests {
17971797

17981798
/// The same, after a removal that leaves surviving relatives behind.
17991799
///
1800-
/// Removing the middle of a chain is the case the closure has to get right:
1801-
/// the parent loses descendants it still has to account for, and the child
1802-
/// loses an ancestor while remaining in the pool.
1800+
/// Every entry in the fixture is removed in turn, from a fresh pool each
1801+
/// time, rather than one chosen entry. Each victim exercises a different
1802+
/// shape: removing the middle of the chain leaves a parent that must forget
1803+
/// descendants and a child that must forget an ancestor, and removing
1804+
/// `leaf` takes the fan-in `joined` with it while `sibling` — `joined`'s
1805+
/// *other* parent — survives and has to drop it from its descendant totals.
1806+
/// That last one is not reachable by removing any single entry the closure
1807+
/// names directly, and is the case a chosen victim is most likely to miss.
1808+
///
1809+
/// The victim is addressed by entry id, which is the slab index and so
1810+
/// follows insertion order. An earlier revision picked it out of
1811+
/// `pool.by_txid`, a `HashMap` whose iteration order is randomized per
1812+
/// process: the test removed a different entry on every run, and under a
1813+
/// mutation that took the closure after the removal instead of before it,
1814+
/// it went red in only 36 of 60 runs.
18031815
#[test]
18041816
fn incremental_metadata_matches_the_full_recompute_after_removals() -> Result<(), MempoolError>
18051817
{
1806-
let (mut pool, _outs) = graph_pool()?;
1818+
for victim in 0..6_u32 {
1819+
let (mut pool, _outs) = graph_pool()?;
1820+
let removed = pool.remove_entry_and_descendants(victim);
1821+
assert!(
1822+
!removed.is_empty(),
1823+
"entry {victim} must be present in the fixture"
1824+
);
1825+
1826+
let incremental = totals(&pool);
1827+
pool.recompute_all_metadata();
1828+
assert_eq!(
1829+
incremental,
1830+
totals(&pool),
1831+
"incremental removal metadata diverged from the full recompute \
1832+
after removing entry {victim}"
1833+
);
1834+
}
1835+
Ok(())
1836+
}
18071837

1808-
// `sibling` (label 4) is the second entry funded by root; removing it
1809-
// takes its own descendant `joined` with it and leaves root and the
1810-
// chain behind.
1811-
let victims = pool.by_txid.values().copied().collect::<Vec<_>>();
1812-
let Some(target) = victims.get(3).copied() else {
1813-
panic!("fixture must hold at least four entries");
1838+
/// The eviction path is a `remove_entries` caller nothing else drives.
1839+
///
1840+
/// `insert_entry` calls `enforce_size_limit` when an acceptance puts the
1841+
/// pool over `max_total_bytes`, and that removes packages the caller never
1842+
/// named. The refresh has to leave both the package totals and the priority
1843+
/// index in the state a full rebuild would, from inside the insertion that
1844+
/// triggered it.
1845+
#[test]
1846+
fn eviction_during_insertion_leaves_metadata_a_rebuild_agrees_with() -> Result<(), MempoolError>
1847+
{
1848+
let limits = MempoolLimits {
1849+
max_total_bytes: 400,
1850+
..MempoolLimits::default()
18141851
};
1815-
let _removed = pool.remove_entry_and_descendants(target);
1852+
let mut pool = Mempool::new(limits);
1853+
1854+
let root = tx(31, vec![OutPoint::null()]);
1855+
let root_out = OutPoint::new(root.compute_txid(), 0);
1856+
pool.insert_entry(MempoolEntry::new(Arc::new(root), 100, 500, 0, 1))?;
1857+
let child = tx(32, vec![root_out]);
1858+
let child_out = OutPoint::new(child.compute_txid(), 0);
1859+
pool.insert_entry(MempoolEntry::new(Arc::new(child), 100, 9_000, 1, 1))?;
1860+
pool.insert_entry(MempoolEntry::new(
1861+
Arc::new(tx(33, vec![child_out])),
1862+
100,
1863+
100_000,
1864+
2,
1865+
1,
1866+
))?;
1867+
pool.insert_entry(MempoolEntry::new(
1868+
Arc::new(tx(34, vec![OutPoint::null()])),
1869+
100,
1870+
200,
1871+
3,
1872+
1,
1873+
))?;
1874+
pool.insert_entry(MempoolEntry::new(
1875+
Arc::new(tx(35, vec![OutPoint::null()])),
1876+
100,
1877+
300,
1878+
4,
1879+
1,
1880+
))?;
1881+
assert!(
1882+
pool.len() < 5,
1883+
"the fixture must actually cross the size limit"
1884+
);
18161885

18171886
let incremental = totals(&pool);
1887+
let index = pool.pareto.top_n(pool.pareto.len()).collect::<Vec<_>>();
18181888
pool.recompute_all_metadata();
18191889
assert_eq!(
18201890
incremental,
18211891
totals(&pool),
1822-
"incremental removal metadata diverged from the full recompute"
1892+
"eviction left package totals a full recompute disagrees with"
1893+
);
1894+
assert_eq!(
1895+
index,
1896+
pool.pareto.top_n(pool.pareto.len()).collect::<Vec<_>>(),
1897+
"eviction left a stale priority index behind"
18231898
);
18241899
Ok(())
18251900
}
@@ -1856,22 +1931,28 @@ mod tests {
18561931
assert_eq!(stats.total_fee, fee, "stats.total_fee wrong after {stage}");
18571932
}
18581933

1859-
let (mut pool, _outs) = graph_pool()?;
1934+
let (mut pool, outs) = graph_pool()?;
18601935
check(&pool, "inserts");
18611936

1862-
let Some(&txid) = pool.by_txid.keys().next() else {
1937+
// Addressed through the fixture's own handles, in insertion order.
1938+
// `pool.by_txid` is a `HashMap`, so picking from it would choose a
1939+
// different subject on every run — see
1940+
// `incremental_metadata_matches_the_full_recompute_after_removals`.
1941+
let Some(root) = outs.first().map(|out| out.txid) else {
18631942
panic!("fixture must hold entries");
18641943
};
1865-
assert!(pool.prioritise(txid, 250_000), "prioritise up must apply");
1944+
assert!(pool.prioritise(root, 250_000), "prioritise up must apply");
18661945
check(&pool, "a positive fee delta");
18671946

18681947
assert!(
1869-
pool.prioritise(txid, -100_000),
1948+
pool.prioritise(root, -100_000),
18701949
"prioritise down must apply"
18711950
);
18721951
check(&pool, "a negative fee delta");
18731952

1874-
let Some(&victim_txid) = pool.by_txid.keys().find(|candidate| **candidate != txid) else {
1953+
// `mid`: removing it takes the rest of the chain with it and leaves
1954+
// both a surviving parent and an unrelated entry behind.
1955+
let Some(victim_txid) = outs.get(1).map(|out| out.txid) else {
18751956
panic!("fixture must hold a second entry");
18761957
};
18771958
let removed = pool.remove_by_txid(&victim_txid);
@@ -1893,15 +1974,10 @@ mod tests {
18931974
/// stale.
18941975
#[test]
18951976
fn prioritise_reindexes_the_descendants_it_lifts() -> Result<(), MempoolError> {
1896-
let (mut pool, _outs) = graph_pool()?;
1977+
let (mut pool, outs) = graph_pool()?;
18971978

18981979
// Lift the chain root, which every entry in the chain descends from.
1899-
let Some((&txid, _)) = pool
1900-
.by_txid
1901-
.iter()
1902-
.min_by_key(|(_, id)| **id)
1903-
.map(|(txid, id)| (txid, id))
1904-
else {
1980+
let Some(txid) = outs.first().map(|out| out.txid) else {
19051981
panic!("fixture must hold entries");
19061982
};
19071983
assert!(pool.prioritise(txid, 5_000_000), "prioritise must apply");

docs/benchmarks/mempool-pareto.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,7 @@ The tests were then audited by mutation:
138138
| the closure forgets descendants | red | 3 tests failed |
139139
| the closure forgets ancestors | red | 2 tests failed |
140140
| a removal takes its closure after the removal | red | 1 test failed |
141+
| eviction skips the priority reindex | red | 3 tests failed |
141142
| descendant totals count only the entry itself | red | 5 tests failed |
142143
| the refresh skips the priority reindex | red | 2 tests failed |
143144
| an insertion forgets the running vsize total | red | 29 tests failed |
@@ -165,6 +166,33 @@ caught it. The oracle now keeps its own verbatim copy of the comparison, and the
165166
same mutation kills all three ordering tests. **An oracle that shares code with
166167
the implementation cannot disagree with it.**
167168

169+
A second review pass found a defect the first audit could not have caught, because
170+
the audit itself was reading a coin flip. `Mempool::by_txid` is a `HashMap`, so its
171+
iteration order is randomized per process. Two of the tests above picked their
172+
subject out of it — `victims.get(3)` and `by_txid.keys().next()` — and therefore
173+
removed or bumped a *different* transaction on every run. Rebuilding the
174+
"a removal takes its closure after the removal" mutant and running the binary 40
175+
times put the row above at **36 red out of 60**: the mutation was reported killed
176+
because the run that happened to be recorded had drawn a victim that exposed it.
177+
178+
Every subject is now addressed by entry id, which is the slab index and so follows
179+
insertion order, and the removal test removes *each* of the six fixture entries in
180+
turn rather than one. Re-run at 40 executions per mutant, every row above is now
181+
40/40 red and the baseline 0/40. The exhaustive sweep also reaches a case no single
182+
chosen victim could: removing `leaf` takes the fan-in `joined` with it while
183+
`sibling``joined`'s *other* parent — survives and has to drop it from its
184+
descendant totals.
185+
186+
`eviction_during_insertion_leaves_metadata_a_rebuild_agrees_with` closes the other
187+
gap the pass found. `enforce_size_limit` is a `remove_entries` caller reached from
188+
inside `insert_entry`, removing packages the caller never named, and no test drove
189+
the refresh through it. It is now among the tests that kill both the
190+
forgets-ancestors and the skips-the-reindex mutations.
191+
192+
**A test that chooses its subject from a `HashMap` is a different test on every
193+
run**, and a mutation audit run once against one cannot distinguish "killed" from
194+
"killed this time".
195+
168196
One measurement artefact, recorded because it briefly read as a coverage gap:
169197
`cargo test -p <crate>` stops at the first failing target by default, so a
170198
mutation that fails the lib suite never runs the integration suites. The

0 commit comments

Comments
 (0)