fix(txpool): prevent removed transactions from being included in blocks

[wo-o]

Problem

A race condition exists where BestTransactions iterator creates an independent snapshot via BTreeMap::clone(). When the maintenance job removes transactions from the pool, existing iterators remain unaware, causing removed transactions to be included in blocks.

Production Impact

During load testing with --txpool.lifetime 600, this caused sequencer failure:

2025-11-15T13:30:01.790071Z WARN The database read transaction has been open for too long.
    open_duration=190.664088693s self.txn_id=7039115

2025-11-15T13:30:01.947723Z WARN Attempt to calculate state root for an old block
    might result in OOM target=2846869

The sequencer stopped producing blocks because:

1. Maintenance removed expired transactions from pool
2. Block builder's snapshot still contained the removed transactions
3. Execution attempted to process non-existent transactions
4. Database read stuck
5. State root calculation failed

This is critical for transactions with time-based expiration.

Root Cause

crates/transaction-pool/src/pool/pending.rs:110 - The iterator snapshot is created via BTreeMap::clone(), which creates an independent copy. The pool has notification support for new transactions but lacks removal notifications.

Solution

Add removal notifications following the existing pattern for new transactions:

1. PendingPool broadcasts transaction removals via new removed_transaction_notifier channel
2. BestTransactions subscribes to removal notifications on creation
3. Iterator processes removals before yielding next transaction

[mattsse]

could you elaborate on why

causing removed transactions to be included in blocks.

would be problematic, because atm I believe this would only happen if the pending pool is at capacity, but I assume you need this to satisfy some protocol level rules?

Block builder's snapshot still contained the removed transactions
Execution attempted to process non-existent transactions
Database read stuck

not following the sequence of events here that's causing db/State root issues, maybe this has something to do with some protocol specific rules?
if the blockbuilder adds this tx into the block, how would this cause execution errors?

[mattsse]

lacking some context on why excluding removed txs is critical

but maybe this is similar to OP interop inclusion rules

reth/crates/optimism/txpool/src/transaction.rs

Lines 104 to 115 in 9f3949c

	impl<Cons, Pooled> MaybeInteropTransaction for OpPooledTransaction<Cons, Pooled> {
	fn set_interop_deadline(&self, deadline: u64) {
	self.interop.store(deadline, Ordering::Relaxed);
	}
	fn interop_deadline(&self) -> Option<u64> {
	let interop = self.interop.load(Ordering::Relaxed);
	if interop > NO_INTEROP_TX {
	return Some(interop)
	}
	None
	}

which is checked during block building for example

reth/crates/optimism/payload/src/builder.rs

Line 696 in 9f3949c

let interop = tx.interop_deadline();

reth/crates/optimism/payload/src/builder.rs

Lines 730 to 737 in 9f3949c

	// We skip invalid cross chain txs, they would be removed on the next block update in
	// the maintenance job
	if let Some(interop) = interop &&
	!is_valid_interop(interop, self.config.attributes.timestamp())
	{
	best_txs.mark_invalid(tx.signer(), tx.nonce());
	continue
	}

[mattsse]

not opposed to this, but since this is an internal channel we can unify this by introducing

enum PendingPoolEvent<T> {Added(Tx),Removed(tx)}