feat(tests): opt-in background activity mode for chains [concept]

[antonbaliasnikov]

Concept — for review, not merge

This is a concept PR exploring a "noisy" run mode for test chains, opened in draft. It's working and fully tested, but the API surface is up for discussion before we commit to it.

Motivation

Today every chain runs in silent mode: after setup it sits idle until a test explicitly calls ping() / send_tx() / transfer(). Tests therefore always run against a perfectly still chain, which hides race conditions and ordering bugs between the sequencer, L1 watcher, priority queue, and batch pipeline.

This adds an opt-in background activity mode: a chain continuously fires L2 self-transfers and L1→L2 deposits in the background while a test runs, so the test executes inside a live, moving chain. Inspired by the production matter-labs/watchdog service (flow-per-activity-type, crash-safe restart, start-to-start pacing).

What's here

New tests/src/activity.rs owns the noise:

• ACTIVITY_WALLET_KEYS — a private pool (Anvil HD accounts feat: test proper migration support #10–fix: enforce Anvil block gas limit to catch oversized server blocks #19), separate from test wallets so the two never share nonces. Funded on L1 via --accounts 20 and on L2 during the fixture's cache-miss path, so the deposits land in the cached snapshot (zero cost on cache hit).
• ActivityConfig — per-flow Option<Duration> (None = disabled). transfers_only() / deposits_only() presets for targeted race tests.
• ActivityHandle — pause() / resume() / stop().await / is_alive() plus txs_sent() / deposits_sent() counters.
• Crash-safe flow loops — bounded error budget (MAX_CONSECUTIVE_ERRORS), then the task sets a failed flag and exits so is_alive() flips (a broken loop is detectable, not silent). Start-to-start interval pacing avoids drift under slow RPC.

Wiring:

• Chain::start_background_activity(config) — double-start guard (panics if already running) + per-chain dedicated L1 deposit wallet (chain_index % pool_len), so no two chains share an L1 signer.
• Ecosystem::start_background_activity(config) — fan-out across all chains, panics if chain_count > pool_len.
• ecosystem fixture gains activity: Option<ActivityConfig> that auto-starts noise from setup.

Usage

// Silent (existing tests — unchanged)
async fn my_test(#[future] ecosystem: Ecosystem) -> Result<()> { ... }

// Noisy from the start
async fn noisy(
    #[future]
    #[with(vec![6565], Some(ActivityConfig::default()))]
    ecosystem: Ecosystem,
) -> Result<()> { ... }

// Manual handle — full control
let h = eco.chain().start_background_activity(ActivityConfig::deposits_only());
h.pause(); /* sensitive assertion */ h.resume();
assert!(h.deposits_sent() >= 1 && h.is_alive());
h.stop().await; // restart-safe; Drop is teardown-only

Semantics worth a look during review

• pause() stops new submissions; an in-flight L1→L2 deposit may still land. Not an exact quiesce.
• stop().await is the only restart-safe teardown; Drop aborts without awaiting.
• txs_sent counts RPC-accepted hashes (mempool), deposits_sent counts L1-confirmed deposits — neither implies mining/finalization.

Testing

• Unit tests for config presets + round-robin helper.
• 5 integration tests: manual-handle lifecycle, restart-after-stop, pause-halts-submissions, fixture-param auto-start, double-start panic.
• Full suite green (16/16), including pre-existing chain_executes_a_batch, two_chains_settle_on_l1, and the v30→v31 upgrade test — no regressions.

Possible follow-ups (out of scope here)

• Extract to a lib/activity crate if we ever want to replace the TS watchdog with a Rust one.
• Withdrawal flows, contract-call activity, Prometheus metrics, mined/finalized counters.

🤖 Generated with Claude Code