You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The clerk entry point and the prover-input orchestration are exercised only through slow end-to-end tests, and several error paths are unreachable without a real prover.
What
Add behavior tests with mockall doubles at the IVC wiring seams (clerk entry point and prover input preparation) to verify orchestration, error mapping and state threading, without re-testing what the proof system module already covers: proof generation and verification stay covered by the existing prover and circuit tests.
How
Paths are relative to mithril-stm/src/.
Introduce one prover trait per proof system, CertificateProver (proof_system/halo2_snark, implemented by SnarkProver) and IvcStepProver (proof_system/ivc_halo2_snark, implemented by IvcProver), with a single-argument IvcStepInput carrying all inputs of an IVC step
Inject the provers into Clerk through a SnarkProverFactory built lazily per aggregation branch, with a test-gated constructor taking the mock and no public API change
Add behavior tests for Clerk::aggregate_signatures_with_type (protocol/aggregate_signature/clerk.rs) with mocked factory and provers, covering input assembly, error mapping and rolling-state threading for the Snark and IvcSnark branches; move the two existing precondition tests onto IvcStepInput::try_new
Cover the certificate-rejection path of IvcProverInput::prepare (proof_system/ivc_halo2_snark/prover_input.rs) in the fast tier with empty proof bytes against the dummy IVC setup
Retire the prover_input.rs slow tests made redundant by the fast tier, promote the two that need no setup, and un-ignore the ones that stay so the nightly slow run executes them
Assess the gain (expected: closes the wiring gaps at zero runtime cost) and report it in the issue
Proposed implementation
Seams: one trait per proof system
Each trait mirrors an existing module and has a single production implementor that already exists. Both are pinned to MithrilMembershipDigest because the clerk already pins it, and both are object safe.
Naming.CertificateProver, IvcStepProver, IvcStepInput, SnarkProverFactory, NonDeterministicSnarkProverFactory, prove_certificate, prove_step and IvcProver::try_new_non_deterministic are working names. They must follow whatever convention #3421 settles on for the recursive and non-recursive stacks, at implementation time if #3421 lands first, or as part of #3421 otherwise.
Non-recursive (proof_system/halo2_snark):
/// Non-recursive proving side: one certificate proof per aggregation.#[cfg_attr(test, mockall::automock)]pub(crate)traitCertificateProver{fnverification_key(&self) -> &NonRecursiveCircuitVerifyingKey;fnprove_certificate(&mutself,clerk:&SnarkClerk,signatures:&[SingleSignature],message:&[u8],) -> StmResult<SnarkProof<MithrilMembershipDigest>>;}impl<R:RngCore + CryptoRng>CertificateProverforSnarkProver<R>{// delegates to aggregate_signatures::<MithrilMembershipDigest> and verification_key}
Recursive (proof_system/ivc_halo2_snark):
/// All inputs of one IVC step.pub(crate)structIvcStepInput{pub(crate)certificate_proof:SnarkProof<MithrilMembershipDigest>,pub(crate)message:Vec<u8>,pub(crate)aggregate_verification_key:AggregateVerificationKeyForSnark<MithrilMembershipDigest>,pub(crate)global:Global,pub(crate)protocol_message_preimage:ProtocolMessagePreimage,pub(crate)genesis_bootstrap:IvcGenesisBootstrapInput,pub(crate)rolling_state:Option<IvcRollingState>,}implIvcStepInput{/// Fails on a missing genesis verifying key, a prover data without IVC rolling state,/// a missing genesis Schnorr signature or a preimage that is not PREIMAGE_SIZE bytes.pub(crate)fntry_new(certificate_proof:SnarkProof<MithrilMembershipDigest>,message:&[u8],aggregate_verification_key:AggregateVerificationKeyForSnark<MithrilMembershipDigest>,ancillary_input:AncillaryProofInput,certificate_verifying_key:&NonRecursiveCircuitVerifyingKey,ivc_verifying_key:&RecursiveCircuitVerifyingKey,) -> StmResult<Self>}/// Recursive proving side: advances the IVC chain by one step.#[cfg_attr(test, mockall::automock)]pub(crate)traitIvcStepProver{fnivc_verifying_key(&self) -> &RecursiveCircuitVerifyingKey;fnprove_step(&mutself,step_input:IvcStepInput,) -> StmResult<(IvcProof<Blake2b256>,Option<IvcRollingState>)>;}impl<R:RngCore + CryptoRng>IvcStepProverforIvcProver<R>{// ivc_verifying_key read from ivc_setup, prove_step unpacks IvcStepInput into prove}
IvcStepInput is the input-preparation half of today's ivc_prover_input_preparation_and_prove. The pure assembly lives in try_new, so the precondition errors are plain unit tests, and the seam takes one owned argument with no lifetime and no clone: message is copied once, the AVK is already returned owned by compute_aggregate_verification_key_for_snark, the certificate proof is moved as it is today, and the rolling state is moved out of the AncillaryProofInput that aggregate_signatures_with_type already receives by value. The move needs two consuming accessors:
The struct cannot exist before the certificate proof, so the clerk keeps main's order: prove the certificate, then validate the ancillary input.
IvcStepProver exposes only the recursive key: the certificate key is the non-recursive prover's, there is no second certificate key. IvcProver keeps its internal ivc_setup.certificate_verifying_key for the in-circuit verifier gadget; the clerk never reads it.
Injection: SnarkProverFactory owned by the clerk
Provers are expensive (trusted setup download, key generation) and must only be built inside the branch that needs them, so the clerk owns a factory rather than the provers. The trait and the production factory live in a new proof_system/snark_prover_factory.rs since they depend on both proof-system modules, exported from proof_system/mod.rs under future_snark (MockSnarkProverFactory under test as well).
/// Builds the provers an aggregation needs, once the aggregate signature type is known.#[cfg_attr(test, mockall::automock)]pub(crate)traitSnarkProverFactory:Debug + Send + Sync{/// Builds the non-recursive prover for the certificate proof.fncertificate_prover(&self,parameters:&Parameters) -> StmResult<Box<dynCertificateProver>>;/// Builds the recursive prover for one IVC step.fnivc_step_prover(&self,parameters:&Parameters) -> StmResult<Box<dynIvcStepProver>>;}/// Production factory: `SnarkProver<OsRng>` and `IvcProver<OsRng>` over the trusted setup.#[derive(Debug,Default)]pub(crate)structNonDeterministicSnarkProverFactory;implSnarkProverFactoryforNonDeterministicSnarkProverFactory{fncertificate_prover(&self,parameters:&Parameters) -> StmResult<Box<dynCertificateProver>>{Ok(Box::new(SnarkProver::try_new_non_deterministic(
parameters,MERKLE_TREE_DEPTH_FOR_SNARK,)?))}fnivc_step_prover(&self,parameters:&Parameters) -> StmResult<Box<dynIvcStepProver>>{Ok(Box::new(IvcProver::try_new_non_deterministic(parameters)?))}}
The IVC side needs the symmetric constructor on IvcProver (proof_system/ivc_halo2_snark/proof.rs), which is step 3 of today's clerk branch moved out of clerk.rs:
implIvcProver<OsRng>{/// Loads the IVC setup from the trusted setup and the recursive key provider/// derived from `parameters` and `MERKLE_TREE_DEPTH_FOR_SNARK`.pub(crate)fntry_new_non_deterministic(parameters:&Parameters) -> StmResult<Self>{let trusted_setup_provider = TrustedSetupProvider::default();let certificate_key_provider =
KeyProvider::for_non_recursive_circuit(parameters,MERKLE_TREE_DEPTH_FOR_SNARK)?;let recursive_key_provider = KeyProvider::for_recursive_circuit(certificate_key_provider);let ivc_setup =
IvcSnarkProverSetup::load(&trusted_setup_provider,&recursive_key_provider)?;Ok(Self{ivc_setup:Arc::new(ivc_setup),rng:OsRng,})}}
The clerk owns the factory; the two public constructors keep their signatures and only gain one field initializer, and a test-gated constructor takes the mock explicitly:
#[cfg(feature = "future_snark")]usecrate::proof_system::{NonDeterministicSnarkProverFactory,SnarkProverFactory};#[cfg(all(feature = "future_snark", test))]usecrate::proof_system::MockSnarkProverFactory;#[derive(Debug,Clone)]pubstructClerk<D:MembershipDigest>{concatenation_proof_clerk:ConcatenationClerk,#[cfg(feature = "future_snark")]snark_proof_clerk:Option<SnarkClerk>,#[cfg(feature = "future_snark")]snark_prover_factory:Arc<dynSnarkProverFactory>,phantom_data:PhantomData<D>,}impl<D:MembershipDigest>Clerk<D>{/// Create a Clerk from a signer.pubfnnew_clerk_from_signer(signer:&Signer<D>) -> Self{Self{concatenation_proof_clerk:ConcatenationClerk::new_clerk_from_signer(signer),#[cfg(feature = "future_snark")]snark_proof_clerk: signer
.closed_key_registration.has_snark_verification_keys().then(|| SnarkClerk::new_clerk_from_signer(signer)),#[cfg(feature = "future_snark")]snark_prover_factory:Arc::new(NonDeterministicSnarkProverFactory),phantom_data:PhantomData,}}/// Create a Clerk from a closed key registration.pubfnnew_clerk_from_closed_key_registration(parameters:&Parameters,closed_registration:&ClosedKeyRegistration,) -> Self{Self{concatenation_proof_clerk:ConcatenationClerk::new_clerk_from_closed_key_registration(
parameters,
closed_registration,),#[cfg(feature = "future_snark")]snark_proof_clerk: closed_registration.has_snark_verification_keys().then(|| {SnarkClerk::new_clerk_from_closed_key_registration(parameters, closed_registration)}),#[cfg(feature = "future_snark")]snark_prover_factory:Arc::new(NonDeterministicSnarkProverFactory),phantom_data:PhantomData,}}/// Create a Clerk from a signer whose SNARK provers come from a mocked factory.#[cfg(all(feature = "future_snark", test))]pub(crate)fnnew_clerk_from_signer_with_mock_prover_factory(signer:&Signer<D>,snark_prover_factory:MockSnarkProverFactory,) -> Self{Self{snark_prover_factory:Arc::new(snark_prover_factory),
..Self::new_clerk_from_signer(signer)}}}
#[automock] generates Debug for mocked traits, so Clerk keeps #[derive(Debug, Clone)].
split into IvcStepInput::try_new (assembly) and IvcProver::prove_step (proving)
IvcSnarkProverSetup::load and key providers in clerk.rs
moved to NonDeterministicSnarkProverFactory and IvcProver::try_new_non_deterministic
certificate key read from IvcSnarkProverSetup
read from the prover that produced the certificate proof, as the Snark branch already does
IvcSnark error context labelled Snark
labelled IvcSnark
build_fast_dummy_ivc_setup in clerk.rs tests
relocated to prover_input.rs tests
The aggregate_signatures_for_* helpers are private and never called from tests: every clerk test goes through aggregate_signatures_with_type.
Test doubles and fixtures
Add mockall = { workspace = true } to mithril-stm/[dev-dependencies] (already pinned at workspace level, used by mithril-common). #[cfg_attr(test, automock)] generates MockSnarkProverFactory, MockCertificateProver and MockIvcStepProver.
Fixture
Source
certificate verifying key
NonRecursiveCircuitVerifyingKey::try_from_bytes(NON_RECURSIVE_CIRCUIT_VERIFICATION_KEY_FOR_PRODUCTION) (already used in circuits/halo2/keys.rs tests)
SnarkProof::new(vec![], ..) against build_fast_dummy_ivc_setup; empty proof bytes fail in prepare_and_check, so the rejection path is reached with no proof generated (replaces slow::prepare_rejects_invalid_snark_proof)
Run: cargo test -p mithril-stm --features future_snark -- --skip slow::
Examples
Rolling-state threading and input assembly, through the public entry point:
remove, replaced by fast prepare_rejects_invalid_certificate_proof
prepare_rejects_invalid_genesis_signature
yes
remove, stale: prepare never verifies the genesis signature, only prepare_genesis does
prepare_rejects_invalid_epoch_transition
yes
remove, redundant with fast rejects_out_of_range_cert_epoch in prover_input_helpers
prepare_rejects_step_counter_overflow
yes
remove, redundant with fast build_next_state::rejects_step_counter_overflow
Commits
Each commit builds warning-free with green tests: a trait commit also wires its clerk branch onto the trait, the factory comes once both traits exist, tests follow.
refactor(stm): extract CertificateProver trait from SnarkProver
refactor(stm): extract IvcStepProver trait and IvcStepInput from IvcProver
refactor(stm): inject SNARK provers into the clerk through SnarkProverFactory
test(stm): add behavior tests for IvcStepInput preparation
test(stm): add behavior tests for SNARK clerk wiring
test(stm): add behavior tests for IVC clerk wiring
test(stm): cover certificate rejection in IVC prover input preparation
Why
The clerk entry point and the prover-input orchestration are exercised only through slow end-to-end tests, and several error paths are unreachable without a real prover.
What
Add behavior tests with mockall doubles at the IVC wiring seams (clerk entry point and prover input preparation) to verify orchestration, error mapping and state threading, without re-testing what the proof system module already covers: proof generation and verification stay covered by the existing prover and circuit tests.
How
Paths are relative to
mithril-stm/src/.CertificateProver(proof_system/halo2_snark, implemented bySnarkProver) andIvcStepProver(proof_system/ivc_halo2_snark, implemented byIvcProver), with a single-argumentIvcStepInputcarrying all inputs of an IVC stepClerkthrough aSnarkProverFactorybuilt lazily per aggregation branch, with a test-gated constructor taking the mock and no public API changeClerk::aggregate_signatures_with_type(protocol/aggregate_signature/clerk.rs) with mocked factory and provers, covering input assembly, error mapping and rolling-state threading for theSnarkandIvcSnarkbranches; move the two existing precondition tests ontoIvcStepInput::try_newIvcProverInput::prepare(proof_system/ivc_halo2_snark/prover_input.rs) in the fast tier with empty proof bytes against the dummy IVC setupprover_input.rsslow tests made redundant by the fast tier, promote the two that need no setup, and un-ignore the ones that stay so the nightly slow run executes themProposed implementation
Seams: one trait per proof system
Each trait mirrors an existing module and has a single production implementor that already exists. Both are pinned to
MithrilMembershipDigestbecause the clerk already pins it, and both are object safe.Non-recursive (
proof_system/halo2_snark):Recursive (
proof_system/ivc_halo2_snark):IvcStepInputis the input-preparation half of today'sivc_prover_input_preparation_and_prove. The pure assembly lives intry_new, so the precondition errors are plain unit tests, and the seam takes one owned argument with no lifetime and no clone:messageis copied once, the AVK is already returned owned bycompute_aggregate_verification_key_for_snark, the certificate proof is moved as it is today, and the rolling state is moved out of theAncillaryProofInputthataggregate_signatures_with_typealready receives by value. The move needs two consuming accessors:The struct cannot exist before the certificate proof, so the clerk keeps main's order: prove the certificate, then validate the ancillary input.
IvcStepProverexposes only the recursive key: the certificate key is the non-recursive prover's, there is no second certificate key.IvcProverkeeps its internalivc_setup.certificate_verifying_keyfor the in-circuit verifier gadget; the clerk never reads it.Injection:
SnarkProverFactoryowned by the clerkProvers are expensive (trusted setup download, key generation) and must only be built inside the branch that needs them, so the clerk owns a factory rather than the provers. The trait and the production factory live in a new
proof_system/snark_prover_factory.rssince they depend on both proof-system modules, exported fromproof_system/mod.rsunderfuture_snark(MockSnarkProverFactoryundertestas well).The IVC side needs the symmetric constructor on
IvcProver(proof_system/ivc_halo2_snark/proof.rs), which is step 3 of today's clerk branch moved out ofclerk.rs:The clerk owns the factory; the two public constructors keep their signatures and only gain one field initializer, and a test-gated constructor takes the mock explicitly:
#[automock]generatesDebugfor mocked traits, soClerkkeeps#[derive(Debug, Clone)].Clerk after the change
ivc_prover_input_preparation_and_proveIvcStepInput::try_new(assembly) andIvcProver::prove_step(proving)IvcSnarkProverSetup::loadand key providers inclerk.rsNonDeterministicSnarkProverFactoryandIvcProver::try_new_non_deterministicIvcSnarkProverSetupSnarkbranch already doesIvcSnarkerror context labelledSnarkIvcSnarkbuild_fast_dummy_ivc_setupinclerk.rstestsprover_input.rstestsThe
aggregate_signatures_for_*helpers are private and never called from tests: every clerk test goes throughaggregate_signatures_with_type.Test doubles and fixtures
Add
mockall = { workspace = true }tomithril-stm/[dev-dependencies](already pinned at workspace level, used bymithril-common).#[cfg_attr(test, automock)]generatesMockSnarkProverFactory,MockCertificateProverandMockIvcStepProver.NonRecursiveCircuitVerifyingKey::try_from_bytes(NON_RECURSIVE_CIRCUIT_VERIFICATION_KEY_FOR_PRODUCTION)(already used incircuits/halo2/keys.rstests)RecursiveCircuitVerifyingKey::try_from_bytes(RECURSIVE_CIRCUIT_VERIFICATION_KEY_FOR_PRODUCTION)SnarkProof::new(vec![], params, MERKLE_TREE_DEPTH_FOR_SNARK)IvcProof::new(IvcProofBytes::empty(), State::genesis(), trivial_accumulator(&[]))build_standard_rolling_statefromprover_input_helpers::testsbuild_fast_dummy_ivc_setup(tiny circuit, unsafe SRS, a couple of seconds of keygen), only for the certificate-rejection test inprover_input.rsShared clerk test helpers (
clerk.rstest module):Test matrix (fast tier,
future_snark)IvcStepInput::try_new(no double):ivc_step_input_fails_when_prover_data_carries_no_ivc_rolling_stateivc_step_input_fails_when_genesis_verification_key_is_absentivc_step_input_fails_when_genesis_signature_is_absentivc_step_input_fails_when_message_preimage_has_wrong_sizeivc_step_input_carries_rolling_state_from_ancillary_prover_dataNonewithout prover data,SomewithAncillaryProverData::IvcSnarkivc_step_input_builds_global_from_genesis_data_and_keysGlobalassemblyClerk::aggregate_signatures_with_type(mock factory injected throughnew_clerk_from_signer_with_mock_prover_factory):concatenation_aggregation_does_not_build_snark_provers.never()snark_aggregation_fails_when_snark_clerk_is_missingMissingSnarkClerkbefore any prover is builtsnark_aggregation_builds_certificate_prover_with_clerk_parameterswithfonparameterssnark_aggregation_propagates_prover_factory_errorsnark_aggregation_returns_verifier_data_from_prover_keySnarkVerifierDataassemblysnark_aggregation_propagates_prover_errorivc_aggregation_fails_when_snark_clerk_is_missing.never()ivc_aggregation_propagates_prover_factory_errorivc_aggregation_does_not_prove_step_on_invalid_ancillary_inputexpect_prove_step().never()ivc_aggregation_passes_certificate_proof_and_message_to_step_proverwithfonstep_input: proof bytes, message, AVK, preimage bytesivc_aggregation_uses_certificate_prover_key_for_global_and_verifier_dataGlobalandIvcVerifierDatacarry the certificate prover's oneivc_aggregation_threads_next_rolling_state_into_prover_dataSome->AncillaryProverData::IvcSnarkivc_aggregation_returns_no_prover_data_on_same_epoch_stepNone->Noneivc_aggregation_propagates_certificate_prover_errorexpect_prove_step().never(), context readsIvcSnarkivc_aggregation_propagates_ivc_prover_errorIvcProverInput::prepare(no double):prepare_rejects_invalid_certificate_proofSnarkProof::new(vec![], ..)againstbuild_fast_dummy_ivc_setup; empty proof bytes fail inprepare_and_check, so the rejection path is reached with no proof generated (replacesslow::prepare_rejects_invalid_snark_proof)Run:
cargo test -p mithril-stm --features future_snark -- --skip slow::Examples
Rolling-state threading and input assembly, through the public entry point:
Error mapping, with the recursive prover pinned to never run:
Slow tests
CI runs
cargo nextest runwithout--run-ignored, so#[ignore]tests never run, not even in the nightly slow run. Inprover_input.rs::slow:prepare_genesis_rejects_invalid_signatureslow::unchanged, it needs no setupprepare_genesis_produces_expected_state_and_witnessprepare_at_genesis_produces_advanced_state_and_witness#[ignore]prepare_at_same_epoch_advances_state_correctly#[ignore]prepare_at_next_epoch_carries_lookahead_protocol_parameters#[ignore]prepare_rejects_invalid_snark_proofprepare_rejects_invalid_certificate_proofprepare_rejects_invalid_genesis_signaturepreparenever verifies the genesis signature, onlyprepare_genesisdoesprepare_rejects_invalid_epoch_transitionrejects_out_of_range_cert_epochinprover_input_helpersprepare_rejects_step_counter_overflowbuild_next_state::rejects_step_counter_overflowCommits
Each commit builds warning-free with green tests: a trait commit also wires its clerk branch onto the trait, the factory comes once both traits exist, tests follow.
refactor(stm): extract CertificateProver trait from SnarkProverrefactor(stm): extract IvcStepProver trait and IvcStepInput from IvcProverrefactor(stm): inject SNARK provers into the clerk through SnarkProverFactorytest(stm): add behavior tests for IvcStepInput preparationtest(stm): add behavior tests for SNARK clerk wiringtest(stm): add behavior tests for IVC clerk wiringtest(stm): cover certificate rejection in IVC prover input preparationtest(stm): retire redundant slow prover input tests