Skip to content

Replace prove_all_scenarios monolith with separated tests #3468

Description

@jpraynaud

Why

Grouping many scenarios into one test to amortize the setup in-process is an anti-pattern: it hides per-case results, defeats per-test retries, and couples unrelated assertions.

prove_all_scenarios measures 1100 to 1394 seconds across the last five nightlies. With the two keygen-only tests of the single-threaded ivc-snark-sequential group it forms an 1830 second serial tail that runs after all 2261 other tests of the future_snark job have finished, and that tail is 59% of the job's 3116 second test step.

Splitting cannot buy wall clock back. The test reports user / real = 7.39, so it already saturates the runner's four vCPUs, and its peak RSS is 11.02 GiB against 16 GB, so the sequential group has to stay and a second concurrent prover test would not fit. Every extra test process is therefore pure added nightly time, roughly 70 to 125 seconds each: promoting all eight helpers with their own setups models at 1882 to 2269 seconds against 1394 today, which is the same overhead that forced the June 2026 consolidation, only smaller now that the disk caches turn a cold keygen into a deserialize.

What

Separate the tests, and pay for the split by removing proving work that is now redundant rather than by adding setup work. The five rejection paths generate no proofs and belong in the fast tier; the six proving invocations of the positive paths cover only four distinct transition-type and transcript pairs, so they reduce to four across two tests (JP approved).

How

Paths are relative to mithril-stm/src/ unless they start with .github/ or .config/.

  • Move four rejection scenarios of proof_system/ivc_halo2_snark/proof.rs into the fast tier of proof_system/ivc_halo2_snark/prover_input.rs as IvcProverInput::prepare tests asserting typed errors instead of is_err(), since all five fail inside prepare, which since Recover the ignored IVC prover input preparation tests #3466 needs no SRS, no keygen and no cache: mismatched certificate message, mismatched aggregate verification key, corrupted previous IVC proof, and mismatched Global
  • Delete the corrupted-certificate-proof rejection scenario, an exact duplicate of prepare_rejects_invalid_snark_proof
  • Promote run_bootstrap_path (three proofs: genesis Poseidon, first non-genesis next-epoch Poseidon, returned next-epoch Blake2b) and run_same_epoch_path (one proof, over the committed chain state's real previous IVC proof and non-degenerate folded accumulator) to standalone #[test] functions sharing a plain context builder, with no OnceLock or LazyLock: nextest gives each test its own process, which is why the June consolidation's cells amortized nothing
  • Delete run_next_epoch_path and its two proving invocations: it contributes no new transition-type and transcript pair, and prepare_at_next_epoch_carries_lookahead_protocol_parameters already pins its next state and next accumulator against the same asset, plus four witness fields it does not check
  • Keep prepare_at_next_epoch_carries_lookahead_protocol_parameters, next_epoch_step_circuit_is_accepted, the positive previous-certificate and previous-IVC-proof circuit checks for next-epoch, and next_epoch_accumulator_passes_check with its tampering canary: they carry the mature next-epoch coverage the reduction relies on. The decomposition holds only because no branch couples transition type, incoming-accumulator provenance and outer transcript together; restore a mature next-epoch proving case if production later introduces transition-specific folding, public-input assembly, or transcript selection
  • Build the slow-test Global and genesis bootstrap input from the committed genesis benchmark fixture, so the prover tests consume only the committed inputs they exercise and no longer depend on asset-generation machinery. This removes real work — neither process reassembles the 3000-signer fixture, Merkle tree and certificate relation — but produced no measurable wall-time change, because the ~1.0 GB recursive proving key deserialize dominates setup
  • Keep the verifying-key transcript-representation assertions that detect drift between the committed assets and freshly generated keys
  • Delete prove_all_scenarios, keeping the two positive tests in mod slow so they stay in the ivc-snark-sequential group (.config/nextest.toml) and watched by the proof.rs entry of .github/workflows/scripts/filter-slow-tests.sh
  • Assess the gain with cold and warm caches, reporting setup and proving phases separately, and report it in the issue

The two keygen-only tests of the serial tail (roughly 413 and 98 seconds) never prove, so the memory ceiling that justifies the sequential group likely does not apply to them; moving only those out needs its own measurement and a separate ticket.

Depends on #3433.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions