refactor(tests): adding KMS test material generator with isolated tests infrastructure by maksymsur · Pull Request #270 · zama-ai/kms

maksymsur · 2025-11-12T09:43:54Z

PR Description

This PR replaces Docker Compose-based integration tests with native isolated tests that run directly without Docker, improving test speed, reliability, and developer experience.

What Changed

Test Migration - Complete List

Migrates all Docker-based tests to native isolated execution:

1. Core Service Library Tests (`core/service/src/client/tests/`)

Centralized Isolated Tests (6 tests):

test_central_health_endpoint_availability_isolated - Health endpoint verification
test_central_close_after_drop_isolated - Server shutdown on drop
test_largecipher_isolated - Large ciphertext handling (slow_tests)
test_insecure_central_dkg_backup_isolated - DKG backup/restore
test_insecure_central_autobackup_after_deletion_isolated - Auto-backup after deletion
nightly_test_insecure_central_crs_backup_isolated - CRS backup/restore (nightly)

Threshold Isolated Tests (14 tests):

test_insecure_dkg_isolated - Insecure DKG with Test params (4 parties)
default_insecure_dkg_isolated - Insecure DKG with Default params (4 parties)
secure_threshold_keygen_isolated - Secure keygen with preprocessing (PRSS)
secure_threshold_keygen_crash_online_isolated - Crash recovery during online phase
secure_threshold_keygen_crash_preprocessing_isolated - Crash recovery during preprocessing
test_insecure_threshold_decompression_keygen_isolated - Decompression key generation
test_threshold_health_endpoint_availability_isolated - Health endpoint verification
test_threshold_close_after_drop_isolated - Server shutdown on drop
test_threshold_shutdown_isolated - Graceful shutdown
nightly_test_insecure_threshold_dkg_backup_isolated - DKG backup/restore (nightly)
nightly_test_insecure_threshold_autobackup_after_deletion_isolated - Auto-backup after deletion (nightly)
test_insecure_threshold_crs_backup_isolated - CRS backup/restore
test_threshold_restore_from_backup_isolated - Threshold backup/restore
test_threshold_restore_from_backup_with_custodian_isolated - Custodian-based restore

Example Tests (4 tests):

test_centralized_isolated_example - Centralized test pattern demo
test_threshold_isolated_example - Threshold test pattern demo (4 parties)
test_different_material_types - Material type validation
test_material_validation - Material existence validation

2. CLI Integration Tests (`core-client/tests/integration_tests.rs`)

Centralized CLI Tests (4 tests):

test_centralized_insecure - Keygen + decryption workflow
test_centralized_crsgen_secure - CRS generation
test_centralized_restore_from_backup - Backup/restore flow
test_centralized_custodian_backup - Custodian backup (5 custodians)

Threshold CLI Tests (11 tests):

test_threshold_insecure - Insecure keygen + decryption with Default FHE params (4 parties, k8s_tests)
test_threshold_concurrent_crs - Concurrent CRS generation (4 parties)
nightly_tests_threshold_sequential_crs - Sequential CRS generation with Default FHE params (4 parties)
test_threshold_restore_from_backup - Backup/restore flow (4 parties)
test_threshold_custodian_backup - Custodian backup (5 custodians, 4 parties)
nightly_tests_threshold_sequential_preproc_keygen - Sequential preprocessing with Test FHE params (PRSS, 4 parties, k8s_tests)
test_threshold_concurrent_preproc_keygen - Concurrent preprocessing (PRSS, 4 parties, k8s_tests)
full_gen_tests_default_threshold_sequential_preproc_keygen - Full keygen with Default FHE params (PRSS, 4 parties, k8s_tests)
full_gen_tests_default_threshold_sequential_crs - Full CRS with Default FHE params (4 parties)
test_threshold_mpc_context_init - MPC context initialization (4 parties, k8s_tests)
test_threshold_mpc_context_switch_6 - MPC context switching (6 parties, k8s_tests)

Test Migration - Key Generation

Migrated all 6 threshold key generation tests to isolated infrastructure:

Migrated Tests:

test_insecure_dkg → test_insecure_dkg_isolated
default_insecure_dkg → default_insecure_dkg_isolated
secure_threshold_keygen_test → secure_threshold_keygen_isolated
secure_threshold_keygen_test_crash_online → secure_threshold_keygen_crash_online_isolated
secure_threshold_keygen_test_crash_preprocessing → secure_threshold_keygen_crash_preprocessing_isolated
test_insecure_threshold_decompression_keygen → test_insecure_threshold_decompression_keygen_isolated

File Reorganization:

key_gen_tests.rs → key_gen_helpers.rs (renamed to reflect purpose as helper library)
Original test functions marked as #[ignore] with migration notes
Helper functions preserved for use by integration tests (nightly, custodian backup, reshare)

Key Features:

Decompression key test: Generates 2 insecure keysets, then secure decompression key between them
Crash recovery tests: Simulate party crashes during preprocessing and online phases
PRSS support: All secure tests properly configured with run_prss: true
Proper configuration: Uses KeySetConfig and KeySetAddedInfo for decompression keys

File Changes

Core Changes:

core-client/tests/integration_test.rs - Deleted (Docker-based)
core-client/tests/integration_tests.rs - Added (Native isolated, 15 CLI tests)
core/service/src/client/tests/centralized/*_isolated.rs - Added (5 tests)
core/service/src/client/tests/threshold/key_gen_tests_isolated.rs - Added (6 keygen tests)
core/service/src/client/tests/threshold/key_gen_helpers.rs - Renamed from key_gen_tests.rs (helper functions only)
core/service/src/client/tests/threshold/misc_tests_isolated.rs - Added (3 tests)
core/service/src/client/tests/threshold/restore_from_backup_tests_isolated.rs - Added (2 tests)
core/service/src/client/tests/threshold/misc_tests.rs - Updated (removed 2 redundant tests)
core/service/src/client/tests/threshold/restore_from_backup_tests.rs - Deleted (all tests redundant)
core/service/src/client/tests/threshold/mod.rs - Updated (module declarations)
core/service/src/client/tests/threshold/common.rs - Added threshold_key_gen_secure_isolated helper
core/service/src/client/tests/isolated_test_example.rs - Added (4 example tests)

Testing Infrastructure:

core/service/src/testing/ - New consolidated testing module
- mod.rs - Module exports and prelude
- prelude.rs - Convenient imports for tests
- setup/centralized.rs - CentralizedTestEnv builder with .with_backup_vault() and .with_custodian_keychain()
- setup/threshold.rs - ThresholdTestEnv builder with helper methods and backup/custodian support
- material/manager.rs - Test material management
- material/spec.rs - Material specifications (TestMaterialSpec, MaterialType, KeyType)
- utils.rs - Test utility functions (health checks, encryption helpers, backup utilities)
- helpers.rs - Additional test helpers (domain_to_msg, create_test_material_manager)
- types.rs - Common test types (ServerHandle, TestResult)
core/service/src/util/key_setup/mod.rs - Backward-compatible re-exports for legacy tests
core/service/src/client/test_tools.rs - Legacy test infrastructure (marked for future refactoring)
tools/generate-test-material/ - Complete pre-generation tool with CLI

CI/CD Changes:

.github/workflows/main.yml - Updated test matrix with required features
.github/workflows/common-testing.yml - Added test material generation + kms-custodian build
Makefile - Added test material generation and isolated test targets:
- generate-test-material-all - Generate all material (testing + default)
- generate-test-material-testing - Generate fast testing material only
- generate-test-material-default - Generate production-like material
- validate-test-material - Validate existing material
- clean-test-material - Clean generated material
- test-isolated - Run all isolated tests
- test-isolated-centralized - Run centralized tests only
- test-isolated-threshold - Run threshold tests only
- test-isolated-integration - Run CLI integration tests
backward-compatibility/generate-v0.13.0/Cargo.toml - Updated to current branch commit

How It Works

Test Execution Flow

Native isolated tests run KMS servers as in-process native Rust processes (no Docker), with automatic lifecycle management:

┌─────────────────────────────────────────────────────────────┐
│ Test Function Execution                                      │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  1. Setup Phase                                              │
│     setup_isolated_centralized_cli_test("my_test")          │
│     ├─ Creates TempDir with isolated material               │
│     ├─ Spawns KMS server (native Rust process)              │
│     │  └─ Server listens on 127.0.0.1:54321 (dynamic port)  │
│     └─ Generates config file pointing to server             │
│                                                              │
│  2. CLI Command Execution                                    │
│     ├─ kms-core-client --config client_config.toml          │
│     ├─ Reads config → connects to 127.0.0.1:54321           │
│     └─ Sends gRPC request to running server                 │
│                                                              │
│  3. Server Processing                                        │
│     ├─ Receives gRPC request                                │
│     ├─ Performs operation (keygen, decrypt, etc.)           │
│     └─ Returns gRPC response                                │
│                                                              │
│  4. Test Validation                                          │
│     └─ assert!(output.status.success())                     │
│                                                              │
│  5. Automatic Cleanup (RAII)                                │
│     ├─ _server dropped → server.assert_shutdown()           │
│     ├─ Sends shutdown signal → server stops                 │
│     ├─ Verifies ports closed                                │
│     └─ material_dir dropped → temp directory deleted        │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Key Points:

Native Process: KMS server runs as Rust process (not Docker container)
Dynamic Ports: Each test gets unique port (no conflicts)
RAII Cleanup: Server and temp directory auto-cleanup via Rust Drop trait
Real gRPC: CLI sends actual gRPC requests to running server
Full Isolation: Each test completely isolated (own server, own material, own temp dir)

Test Architecture

We have two complementary test architectures:

1. CLI Integration Tests (`core-client/tests/integration_tests.rs`)

Tests the actual CLI binary by spawning KMS servers and running CLI commands:

#[tokio::test]
async fn test_centralized_insecure() -> Result<()> {
    // Setup: Creates isolated temp dir with pre-generated material
    // Returns: (TempDir, ServerHandle, PathBuf)
    let (material_dir, _server, config_path) =
        setup_isolated_centralized_cli_test("centralized_insecure").await?;
    //   ^^^^^^^^^^^^  ^^^^^^^ ^^^^^^^^^^^
    //   |             |       └─ Path to CLI config file (for --config flag)
    //   |             └─ Running KMS server (auto-shutdown on drop)
    //   └─ Isolated temp directory with test material (auto-cleanup)
    
    let keys_folder = material_dir.path();
    
    // Test: Run actual CLI commands against running server
    let key_id = insecure_key_gen_isolated(&config_path, keys_folder).await?;
    integration_test_commands_isolated(&config_path, keys_folder, key_id).await?;
    
    Ok(())  // Automatic cleanup via RAII
}

Use case: Testing CLI functionality end-to-end (binary + server interaction)

2. Library Tests (`core/service/src/client/tests/`)

Tests the KMS library directly using the builder pattern and gRPC clients:

use kms_lib::testing::prelude::*;

#[tokio::test]
async fn test_insecure_dkg_isolated() -> Result<()> {
    // Setup: Build test environment with builder pattern
    let env = ThresholdTestEnv::builder()
        .with_test_name("insecure_dkg")
        .with_party_count(4)
        .with_threshold(2)
        .with_backup_vault()        // Optional: Enable backup vault
        .with_custodian_keychain()  // Optional: Enable custodian keychain
        .build()
        .await?;
    
    // Test: Use gRPC clients directly (no CLI)
    let mut keygen_tasks = JoinSet::new();
    for client in env.all_clients() {
        let mut cur_client = client.clone();
        keygen_tasks.spawn(async move {
            cur_client.key_gen(Request::new(keygen_req)).await
        });
    }
    
    // Cleanup: Automatic via RAII
    for server in env.into_servers() {
        server.assert_shutdown().await;
    }
    
    Ok(())
}

Use case: Testing library functionality, crash scenarios, and complex multi-party protocols

Builder Features:

.with_backup_vault() - Creates backup storage for all parties
.with_custodian_keychain() - Enables custodian-based key recovery
.with_prss() - Enables PRSS for secure distributed operations
.with_party_count(n) - Configures number of threshold parties
.with_threshold(t) - Sets threshold value for reconstruction

Key Features (Both Approaches):

Native Process: KMS servers run as Rust processes (not Docker containers)
Isolated Material: Each test gets its own temporary directory with test material
Dynamic Ports: Servers use dynamically allocated ports (no conflicts between parallel tests)
RAII Cleanup: Servers and temp directories automatically cleaned up via Rust Drop trait

Threshold Tests (Multi-Party):

For threshold tests, the setup returns a HashMap<u32, ServerHandle> instead of a single ServerHandle:

let (material_dir, _servers, config_path) =
    setup_isolated_threshold_cli_test("threshold_test").await?;
//                   ^^^^^^^^
//                   HashMap<u32, ServerHandle> - one server per party
//                   - Key: party_id (1, 2, 3, 4 for 4-party setup)
//                   - Value: ServerHandle for that party's server
//                   - All servers run simultaneously as native processes
//                   - Each party has unique ports (service + MPC)
//                   - All auto-shutdown when HashMap is dropped

Each party runs as a separate native process with its own ports, simulating a real distributed threshold setup.

┌─────────────────────────────────────────────────────────────────┐
│ tokio runtime is used for simulation                             │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Main Test Task                                                  │
│  ├─ setup_threshold_isolated(4 parties)                         │
│  │                                                               │
│  └─ Spawns 4 async tasks (tokio::spawn):                        │
│      ├─ Task 1: Party 1 Server (service port + MPC port)        │
│      ├─ Task 2: Party 2 Server (service port + MPC port)        │
│      ├─ Task 3: Party 3 Server (service port + MPC port)        │
│      └─ Task 4: Party 4 Server (service port + MPC port)        │
│                                                                  │
│  All tasks run concurrently on tokio thread pool                │
│  (NOT separate OS threads - async tasks)                       │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Test Material Generation

The test infrastructure includes a comprehensive pre-generation system that creates all cryptographic material needed for tests.

Test Material Tool (`tools/generate-test-material/`)

A standalone CLI tool that generates cryptographic material for isolated tests:

Key Features:

Material Types:
- testing - Fast generation with Test parameters (default for local dev)
- default - Production-like generation with Default parameters (slower, for comprehensive testing)
- all - Generates both testing and default material
Key Types Generated:
- ClientKeys - FHE client keys
- SigningKeys - Client signing keys
- ServerSigningKeys - Server signing keys
- FheKeys - Complete FHE key sets
- CrsKeys - Common Reference String keys
- PublicKeys - Public key components
- DecompressionKeys - Decompression keys (slow_tests only)

Note: PRSS (Pseudo-Random Secret Sharing) setup is not pre-generated. It's created at runtime when KMS servers start with run_prss: true in the test configuration.

Party Configurations:
- Centralized: 1 party
- Threshold: 4 parties (threshold=2)
- Custom: Configurable party count and threshold

Makefile Targets

# Generate all material (testing + default)
make generate-test-material-all

# Generate fast testing material only (recommended for local dev)
make generate-test-material-testing

# Generate production-like material with Default parameters
make generate-test-material-default

# Validate existing material
make validate-test-material

# Clean generated material
make clean-test-material

Tool Usage

# Direct cargo usage
cargo run -p generate-test-material -- [OPTIONS]

# Options:
#   --output <DIR>     Output directory (default: ./test-material)
#   --verbose          Enable verbose output
#   <COMMAND>          testing | default | all | validate | clean

# Examples:
cargo run -p generate-test-material -- --output ./test-material --verbose testing
cargo run -p generate-test-material --features slow_tests -- --output ./test-material all

CI Integration

CI generates test material before running tests to avoid generation overhead:

# Step 1: Generate test material (runs once)
- name: Generate Test Material
  working-directory: .  # Must run from workspace root
  run: cargo run -p generate-test-material -- --output ./test-material --verbose testing

# Step 2: Build required binaries
- name: Build kms-custodian binary
  run: cargo build --package kms --bin kms-custodian

# Step 3: Run tests (uses pre-generated material)
- name: Run Tests
  run: cargo nextest run --profile ci --no-fail-fast ${ARGS_TESTS}

Material Structure

test-material/
├── centralized/
│   └── testing/           # Fast test parameters
│       ├── client_keys/
│       ├── signing_keys/
│       ├── server_signing_keys/
│       └── fhe_keys/
└── threshold/
    ├── testing/           # Fast test parameters
    │   ├── client_keys/
    │   ├── signing_keys/
    │   ├── server_signing_keys/
    │   ├── fhe_keys/
    │   ├── crs_keys/
    │   └── public_keys/
    └── default/           # Production-like parameters (slow_tests)
        └── ... (same structure)

Note: PRSS setup is not stored in test-material. It's generated at runtime by KMS servers when tests configure run_prss: true.

Testing Infrastructure Architecture

`core/service/src/testing/` - Test Infrastructure

Type: Library module within kms crate
Purpose: Provides test utilities, helpers, and setup code
Contents:
- setup/centralized.rs - CentralizedTestEnv builder with backup vault and custodian keychain support
- setup/threshold.rs - ThresholdTestEnv builder with helper methods:
  - all_clients() - Iterate all clients
  - all_clients_except(party) - Exclude parties for crash simulation
  - all_servers(), into_servers() - Server iteration and consumption
  - client(party_id), server(party_id) - Access specific parties
- material/manager.rs - Test material copying and management
- material/spec.rs - Material specifications
- utils.rs - Test helper functions
- helpers.rs - Additional test helpers
- types.rs - Common test types
- prelude.rs - Convenient imports for tests
Feature Gate: #[cfg(all(feature = "non-wasm", any(test, feature = "testing")))]
Usage: use kms_lib::testing::prelude::*;

Example:

#[tokio::test]
async fn my_test() -> Result<()> {
    let env = ThresholdTestEnv::builder()
        .with_test_name("my_test")
        .with_party_count(4)
        .with_backup_vault()        // Optional
        .with_custodian_keychain()  // Optional
        .build()
        .await?;
    
    for client in env.all_clients() {
        // test code
    }
    
    Ok(())
}

CI Configuration

Updated test matrix to include required features and proper test filtering:

Integration Tests (2 parallel jobs):

# Threshold tests
--features k8s_tests,testing -- threshold --skip centralized --skip full_gen_tests --skip nightly --skip k8s_ --skip isolated_test_example

# Centralized tests  
--features k8s_tests,testing -- centralized --skip threshold --skip full_gen_tests --skip nightly --skip k8s_ --skip isolated_test_example

Feature Flags Explained:

k8s_tests - PRSS Test Gating
- Purpose: Gates tests requiring PRSS (Pseudo-Random Secret Sharing) network coordination
- Why needed: PRSS tests require sequential execution and stable network environment
- Usage: #[cfg_attr(not(feature = "k8s_tests"), ignore)] on PRSS tests
- When enabled: Only in K8s CI environment with proper network isolation
- Affected tests: 5 PRSS tests (keygen, preprocessing, MPC context)
testing - Test Helper Functions
- Purpose: Enables test-only helper functions and utilities
- Why needed: Security - prevents test utilities from being compiled in production
- Usage: #[cfg(any(test, feature = "testing"))] on helper modules
- Examples: TestMaterialManager, setup_isolated_* functions, test material generation
- Required for: Compiling test code that uses feature-gated functions
non-wasm - Non-WebAssembly Builds
- Purpose: Gates code that cannot compile to WebAssembly
- Why needed: testing module uses tokio, tonic, file I/O (not available in WASM)
- Usage: #[cfg(all(feature = "non-wasm", any(test, feature = "testing")))]
- When enabled: Automatically enabled for all non-WASM builds (default)
- When disabled: Only for WASM builds (wasm32-unknown-unknown target)
Feature flags - Two scenarios:

Scenario A: Run all tests (including PRSS):
```
# Both features required
cargo test --test integration_tests --features k8s_tests,testing -- --test-threads=1
```
- k8s_tests: Enables PRSS tests (5 tests with network coordination)
- testing: Compiles test helper functions
- --test-threads=1: Sequential execution for PRSS tests
Scenario B: Run non-PRSS tests only:
```
# Only testing feature needed (PRSS tests will be ignored)
cargo test --test integration_tests --features testing
```
- PRSS tests automatically skipped (gated by k8s_tests)
- Faster execution, can run in parallel
Why both features?
- Without testing: Test code won't compile (helper functions are feature-gated)
- Without k8s_tests: PRSS tests will be ignored (safe for local development)
- With both: Full test suite runs (CI environment)
Party counts: Most tests use 4 parties, one test uses 6 parties (MPC context switch)

Other Skips:

isolated_test_example - Demonstration tests (not part of test suite)
k8s_ - Kubernetes cluster tests (run separately in kind-testing.yml)
nightly - Slow comprehensive tests (run only in scheduled builds)
full_gen_tests - Full parameter tests (run only in scheduled builds)

Unit Tests:

--features k8s_tests,testing -- --skip centralized --skip threshold --skip isolated_test_example --skip k8s_

Nightly Tests (comprehensive, scheduled only):

--features k8s_tests,testing -- --skip k8s_ --skip isolated_test_example

Runs ALL tests including nightly_* and full_gen_tests_*
No filtering by test category (centralized/threshold)

Running Tests

Local Development

# Generate test material (required first time)
make generate-test-material-testing

# Run all tests
make test-isolated

# Run specific categories
make test-isolated-centralized   # Centralized tests only
make test-isolated-threshold     # Threshold tests only
make test-isolated-integration   # Integration tests only

# Run with PRSS tests (requires k8s_tests feature)
cargo test --test integration_tests --features k8s_tests,testing -- --test-threads=1

Developer Guide

Writing New Tests

Centralized Test:

#[tokio::test]
async fn test_my_feature() -> Result<()> {
    // Setup isolated centralized KMS server
    // test_name: Used for logging/debugging (appears in tracing output)
    // Returns: (TempDir, ServerHandle, PathBuf)
    let (material_dir, _server, config_path) =
        setup_isolated_centralized_cli_test("my_test").await?;
    //                                      ^^^^^^^^^ 
    //                                      Test identifier for logging/debugging
    //   ^^^^^^^^^^^^  ^^^^^^^ ^^^^^^^^^^^
    //   |             |       └─ Path to CLI config file (for --config flag)
    //   |             └─ Running KMS server (auto-shutdown on drop)
    //   └─ Isolated temp directory with test material (auto-cleanup)
    
    // Run CLI commands against the isolated KMS server
    let output = Command::new(env!("CARGO_BIN_EXE_kms-core-client"))
        .args(["--config", config_path.to_str().unwrap()])
        .args(["my-command"])
        .output()?;
    
    assert!(output.status.success());
    Ok(())
}

Threshold Test:

#[tokio::test]
async fn test_my_threshold_feature() -> Result<()> {
    // Setup isolated threshold KMS cluster (4 parties, Default FHE params)
    // Returns: (TempDir, HashMap<u32, ServerHandle>, PathBuf)
    let (material_dir, _servers, config_path) =
        setup_isolated_threshold_cli_test_default("my_test", 4).await?;
    //   ^^^^^^^^^^^^  ^^^^^^^^ ^^^^^^^^^^^
    //   |             |        └─ Path to CLI config file (for --config flag)
    //   |             └─ HashMap of 4 running KMS servers (party_id -> ServerHandle)
    //   └─ Isolated temp directory with test material (auto-cleanup)
    
    // Run CLI commands against threshold KMS cluster
    // CLI automatically communicates with all 4 parties via config
    Ok(())
}

PRSS Test (Sequential, K8s CI Only):

#[cfg(feature = "k8s_tests")]  // Clean feature gating - only compiles with k8s_tests
#[tokio::test]
#[serial]  // Sequential execution required (PRSS network coordination)
async fn test_my_prss_feature() -> Result<()> {
    // Setup isolated threshold KMS cluster with PRSS enabled
    // Returns: (TempDir, HashMap<u32, ServerHandle>, PathBuf)
    let (material_dir, _servers, config_path) =
        setup_isolated_threshold_cli_test_with_prss("my_test", 4).await?;
    //   ^^^^^^^^^^^^  ^^^^^^^^ ^^^^^^^^^^^
    //   |             |        └─ Path to CLI config file
    //   |             └─ HashMap of 4 KMS servers with PRSS initialized
    //   └─ Isolated temp directory with PRSS material
    
    // Run preprocessing/keygen operations requiring PRSS
    // PRSS enables secure distributed key generation
    Ok(())
}

Library Test (Direct API, No CLI):

use kms_lib::testing::prelude::*;

#[tokio::test]
async fn test_my_library_feature() -> Result<()> {
    // Setup using builder pattern
    let env = ThresholdTestEnv::builder()
        .with_test_name("my_test")
        .with_party_count(4)
        .with_threshold(2)
        .with_prss()  // Enable PRSS if needed
        .build()
        .await?;
    
    // Use helper methods for clear, logical code
    for client in env.all_clients() {
        // Test all parties
    }
    
    // Crash simulation
    let crashed_party = 2;
    for client in env.all_clients_except(crashed_party) {
        // Test with one party crashed
    }
    
    // Cleanup is automatic via RAII
    for server in env.into_servers() {
        server.assert_shutdown().await;
    }
    
    Ok(())
}

Issue ticket number and link

PR Checklist

I attest that all checked items are satisfied. Any deviation is clearly justified above.

Dependency Update Questionnaire (only if deps changed or added)

Answer in the Cargo.toml next to the dependency (or here if updating):

Ownership changes or suspicious concentration?
Low popularity?
Unusual version jump?
Lacking documentation?
Missing CI?
No security / disclosure policy?
Significant size increase?

More details and explanations for the checklist and dependency updates can be found in CONTRIBUTING.md

github-actions · 2025-11-25T10:00:54Z

Consolidated Tests Results 2026-01-07 - 12:57:24

Test Results

1 failed
681 passed

Details

682 tests
not captured
junit-to-ctrf
main test-reporter #2211
refactor(tests): adding KMS test material generator with isolated tests infrastructure #270

Failed Tests

kms client::tests::centralized::misc_tests_isolated::test_largecipher_isolated

test-reporter: Run #2211

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
682	681	1	0	0	0	0	not captured

❌ Some tests failed!

Name	Failure Message
❌ client::tests::centralized::misc_tests_isolated::test_largecipher_isolated	thread 'client::tests::centralized::misc_tests_isolated::test_largecipher_isolated' (52433) panicked at core/service/src/client/tests/centralized/misc_tests_isolated.rs:243:5

Tests

View All Tests

Test Name	Status	Duration
backup::custodian::tests::internal_custodian_context_duplicate_role_should_fail	✅	5ms
backup::custodian::tests::internal_custodian_context_role_greater_than_nodes_should_fail	✅	4ms
backup::custodian::tests::internal_custodian_context_zero_role_should_fail	✅	4ms
backup::operator::tests::operator_new_fails_with_bad_n_t	✅	4ms
backup::operator::tests::operator_new_fails_with_duplicate_roles	✅	5ms
backup::operator::tests::operator_new_fails_with_insufficient_messages	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_header	✅	5ms
backup::operator::tests::operator_new_fails_with_invalid_role	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_timestamp_future	✅	5ms
backup::operator::tests::operator_new_fails_with_invalid_timestamp_past	✅	4ms
backup::operator::tests::operator_new_fails_with_not_enough	✅	4ms
backup::operator::tests::operator_new_fails_with_zero_n	✅	4ms
backup::operator::tests::operator_new_fails_with_zero_t	✅	4ms
backup::operator::tests::operator_timestamp_validation	✅	4ms
backup::operator::tests::validate_recovery_validation_material	✅	5ms
backup::secretsharing::pkcs7::tests::padding_sunshine	✅	4ms
backup::secretsharing::pkcs7::tests::padding_wrong_data	✅	4ms
backup::secretsharing::tests::sharing_missing_shares	✅	91ms
backup::secretsharing::tests::sharing_no_error	✅	148ms
backup::secretsharing::tests::sharing_randomness_test	✅	14.4s
backup::secretsharing::tests::sharing_too_many_missing_shares	✅	11ms
backup::secretsharing::tests::sharing_wrong_params	✅	5ms
backup::secretsharing::tests::sharing_wrong_shares	✅	12ms
backup::seed_phrase::tests::difference	✅	4ms
backup::seed_phrase::tests::mnemonic_robustness	✅	5ms
backup::seed_phrase::tests::sunshine	✅	5ms
backup::tests::custodian_reencrypt	✅	8ms
backup::tests::full_flow::case_1	✅	40ms
backup::tests::full_flow::case_2	✅	27ms
backup::tests::full_flow::case_3	✅	73ms
backup::tests::full_flow_drop_msg	✅	30ms
backup::tests::full_flow_malicious_custodian_init	✅	9ms
backup::tests::full_flow_malicious_custodian_not_enough	✅	12ms
backup::tests::full_flow_malicious_custodian_second	✅	54ms
backup::tests::full_flow_malicious_operator	✅	27ms
backup::tests::operator_setup	✅	7ms
client::crs_gen::tests::process_distributed_crs_result_invalid_signature_does_not_insert_key	✅	8.2s
client::crs_gen::tests::verify_pp_with_tfhers	✅	14.9s
client::tests::centralized::crs_gen_tests::test_crs_gen_centralized	✅	6m 23s
client::tests::centralized::crs_gen_tests::test_crs_gen_manual	✅	1.9s
client::tests::centralized::crs_gen_tests::test_insecure_crs_gen_centralized	✅	9.0s
client::tests::centralized::custodian_backup_tests::test_auto_update_backups_central	✅	4.6s
client::tests::centralized::custodian_backup_tests::test_backup_after_crs_central	✅	11.0s
client::tests::centralized::custodian_backup_tests::test_decrypt_after_recovery_central	✅	5.9s
client::tests::centralized::custodian_backup_tests::test_decrypt_after_recovery_centralized_negative	✅	4.7s
client::tests::centralized::custodian_context_tests::test_new_custodian_context_central	✅	4.1s
client::tests::centralized::key_gen_tests::default_decompression_key_gen_centralized	✅	1m 24s
client::tests::centralized::key_gen_tests::default_key_gen_centralized	✅	38.3s
client::tests::centralized::key_gen_tests::test_decompression_key_gen_centralized	✅	1m 23s
client::tests::centralized::key_gen_tests::test_key_gen_centralized	✅	1.7s
client::tests::centralized::misc_tests_isolated::test_central_close_after_drop_isolated	✅	3.6s
client::tests::centralized::misc_tests_isolated::test_central_health_endpoint_availability_isolated	✅	3.5s
client::tests::centralized::misc_tests_isolated::test_largecipher_isolated	❌	47.1s
client::tests::centralized::public_decryption_tests::default_decryption_centralized::case_1	✅	4.9s
client::tests::centralized::public_decryption_tests::default_decryption_centralized_precompute_sns::case_1	✅	6.0s
client::tests::centralized::public_decryption_tests::test_decryption_central	✅	1.7s
client::tests::centralized::public_decryption_tests::test_decryption_central_no_decompression	✅	1.6s
client::tests::centralized::public_decryption_tests::test_decryption_central_precompute_sns	✅	1.7s
client::tests::centralized::restore_from_backup_tests_isolated::test_insecure_central_autobackup_after_deletion_isolated	✅	4.2s
client::tests::centralized::restore_from_backup_tests_isolated::test_insecure_central_dkg_backup_isolated	✅	4.2s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized::secure_1_true	✅	4.8s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized::secure_2_false	✅	4.7s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_no_compression::secure_1_true	✅	4.5s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_no_compression::secure_2_false	✅	4.5s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_precompute_sns::secure_1_true::compression_1_true	✅	5.7s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_precompute_sns::secure_1_true::compression_2_false	✅	5.6s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_precompute_sns::secure_2_false::compression_1_true	✅	5.7s
client::tests::centralized::user_decryption_tests::default_user_decryption_centralized_precompute_sns::secure_2_false::compression_2_false	✅	5.7s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized::secure_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized::secure_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_1_true::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_1_true::compression_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_2_false::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_2_false::compression_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_1_true::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_1_true::compression_2_false	✅	1.7s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_2_false::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_2_false::compression_2_false	✅	1.6s
client::tests::common::num_blocks_sunshine	✅	5ms
client::tests::isolated_test_example::test_material_validation	✅	6ms
client::tests::isolated_test_example::tests::test_material_spec_creation	✅	4ms
client::tests::isolated_test_example::tests::test_material_spec_serialization	✅	4ms
conf::tests::test_centralized_config	✅	5ms
cryptography::decompression::test::test_1024b	✅	3.2s
cryptography::decompression::test::test_128b	✅	1.6s
cryptography::decompression::test::test_16b	✅	1.4s
cryptography::decompression::test::test_2048b	✅	5.1s
cryptography::decompression::test::test_256b	✅	1.8s
cryptography::decompression::test::test_32b	✅	1.5s
cryptography::decompression::test::test_4b	✅	1.4s
cryptography::decompression::test::test_512b	✅	2.3s
cryptography::decompression::test::test_64b	✅	1.5s
cryptography::decompression::test::test_8b	✅	1.5s
cryptography::decompression::test::test_bad_ciphertext	✅	1.2s
cryptography::decompression::test::test_bad_fhe_type	✅	1.5s
cryptography::decompression::test::test_bool	✅	1.5s
cryptography::decompression::test::test_full_chain_client_copro_kms_uint8::case_1	✅	35ms
cryptography::decompression::test::test_full_chain_client_copro_kms_uint8::case_2	✅	1.7s
cryptography::decompression::test::test_tolerate_non_compressed	✅	495ms
cryptography::encryption::tests::nested_pke_sunshine	✅	6ms
cryptography::encryption::tests::pke_wrong_ct_enc	✅	6ms
cryptography::encryption::tests::pke_wrong_kem_key	✅	5ms
cryptography::hybrid_ml_kem::tests::pke_sunshine	✅	38ms
cryptography::hybrid_ml_kem::tests::pke_wrong_ct_hybrid	✅	34ms
cryptography::hybrid_ml_kem::tests::pke_wrong_kem	✅	33ms
cryptography::hybrid_ml_kem::tests::pke_wrong_key	✅	42ms
cryptography::hybrid_ml_kem::tests::pke_wrong_nonce	✅	34ms
cryptography::hybrid_ml_kem::tests::test_pke_serialize_size	✅	5ms
cryptography::hybrid_ml_kem::tests::validate_consistent_cipher_encoding	✅	4ms
cryptography::signatures::tests::bad_dsep	✅	5ms
cryptography::signatures::tests::bad_signature	✅	5ms
cryptography::signatures::tests::plain_signing	✅	4ms
cryptography::signatures::tests::regression_consistent_enc	✅	4ms
cryptography::signatures::tests::sunshine_verf_key_legacy_serialization	✅	4ms
cryptography::signatures::tests::unnormalized_signature	✅	5ms
cryptography::signatures::tests::validate_zeroize_signing_key	✅	5ms
cryptography::signcryption::tests::bad_signcryption	✅	6ms
cryptography::signcryption::tests::incorrect_server_verf_key	✅	4ms
cryptography::signcryption::tests::signcryption_with_bad_link	✅	5ms
cryptography::signcryption::tests::sunshine	✅	6ms
cryptography::signcryption::tests::sunshine_encoding_decoding	✅	5ms
cryptography::signcryption::tests::test_signcryption_payload_v0_serialization_locked	✅	4ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_custodian_role	✅	6ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_operator_role	✅	6ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_signature	✅	6ms
engine::backup_operator::tests::test_filter_custodian_data_missing_verification_key	✅	6ms
engine::backup_operator::tests::test_filter_custodian_missing_cus_output	✅	6ms
engine::base::tests::sunshine_plaintext_as_u256	✅	4ms
engine::base::tests::test_abi_encoding_fhevm_ebytes	✅	4ms
engine::base::tests::test_compute_external_signature_preproc	✅	6ms
engine::base::tests::test_compute_info_crs	✅	667ms
engine::base::tests::test_compute_info_standard_keygen	✅	111ms
engine::base::tests::test_compute_pt_message_hash	✅	4ms
engine::base::tests::test_deserialize_ciphertext_missing_decompression_key	✅	88ms
engine::base::tests::test_deserialize_ciphertext_wrong_ct_format	✅	78ms
engine::base::tests::test_deserialize_ciphertext_wrong_type	✅	76ms
engine::centralized::central_kms::tests::decrypt_with_bad_client_key	✅	184ms
engine::centralized::central_kms::tests::multiple_default_keys_decrypt	✅	5.9s
engine::centralized::central_kms::tests::multiple_default_keys_user_decrypt	✅	5.8s
engine::centralized::central_kms::tests::multiple_test_keys_access	✅	29ms
engine::centralized::central_kms::tests::multiple_test_keys_decrypt	✅	46ms
engine::centralized::central_kms::tests::multiple_test_keys_user_decrypt	✅	47ms
engine::centralized::central_kms::tests::sanity_check_sns_compression_test_params	✅	55ms
engine::centralized::central_kms::tests::sunshine_default_decrypt	✅	6.0s
engine::centralized::central_kms::tests::sunshine_default_user_decrypt	✅	5.8s
engine::centralized::central_kms::tests::sunshine_test_decrypt	✅	48ms
engine::centralized::central_kms::tests::sunshine_test_user_decrypt	✅	47ms
engine::centralized::central_kms::tests::test_gen_keys	✅	20.8s
engine::centralized::central_kms::tests::user_decrypt_with_bad_client_key	✅	53ms
engine::centralized::central_kms::tests::user_decrypt_with_bad_ephemeral_key	✅	47ms
engine::centralized::central_kms::tests::user_decrypt_with_bad_sig_key	✅	47ms
engine::centralized::service::crs_gen::tests::already_exists	✅	21.0s
engine::centralized::service::crs_gen::tests::invalid_argument	✅	6ms
engine::centralized::service::crs_gen::tests::not_found	✅	6ms
engine::centralized::service::crs_gen::tests::resource_exhausted	✅	6ms
engine::centralized::service::crs_gen::tests::sunshine	✅	569ms
engine::centralized::service::decryption::test_user_decryption::already_exists	✅	21.1s
engine::centralized::service::decryption::test_user_decryption::invalid_argument	✅	96ms
engine::centralized::service::decryption::test_user_decryption::not_found	✅	92ms
engine::centralized::service::decryption::test_user_decryption::resource_exhausted	✅	92ms
engine::centralized::service::decryption::test_user_decryption::sunshine	✅	92ms
engine::centralized::service::decryption::tests_public_decryption::already_exists	✅	95ms
engine::centralized::service::decryption::tests_public_decryption::invalid_argument	✅	93ms
engine::centralized::service::decryption::tests_public_decryption::not_found	✅	96ms
engine::centralized::service::decryption::tests_public_decryption::resource_exhausted	✅	91ms
engine::centralized::service::decryption::tests_public_decryption::sunshine	✅	93ms
engine::centralized::service::initiator::tests::already_exists	✅	6ms
engine::centralized::service::initiator::tests::invalid_argument	✅	6ms
engine::centralized::service::initiator::tests::sunshine	✅	6ms
engine::centralized::service::key_gen::tests::already_exists	✅	84ms
engine::centralized::service::key_gen::tests::invalid_argument	✅	12ms
engine::centralized::service::key_gen::tests::not_found	✅	85ms
engine::centralized::service::key_gen::tests::resource_exhausted	✅	6ms
engine::centralized::service::key_gen::tests::sunshine	✅	87ms
engine::centralized::service::preprocessing::tests::already_exists	✅	6ms
engine::centralized::service::preprocessing::tests::invalid_argument	✅	6ms
engine::centralized::service::preprocessing::tests::not_found	✅	5ms
engine::centralized::service::preprocessing::tests::resource_exhausted	✅	5ms
engine::centralized::service::preprocessing::tests::sunshine	✅	6ms
engine::context::tests::parse_software_version	✅	4ms
engine::context::tests::test_context_info_duplicate_party_ids	✅	5ms
engine::context::tests::test_software_version_display	✅	4ms
engine::context::tests::test_software_version_equality	✅	4ms
engine::context::tests::test_software_version_major_comparison	✅	4ms
engine::context::tests::test_software_version_minor_comparison	✅	4ms
engine::context::tests::test_software_version_no_tag	✅	4ms
engine::context::tests::test_software_version_patch_comparison	✅	4ms
engine::context::tests::test_software_version_unordered_tag	✅	4ms
engine::context_manager::tests::test_custodian_context	✅	10ms
engine::context_manager::tests::test_gen_recovery_request_payloads	✅	8ms
engine::context_manager::tests::test_kms_context	✅	6ms
engine::context_manager::tests::test_kms_context_load_from_storage	✅	6ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_missing_added_info	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_with_added_info_missing_ids	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_with_added_info_with_ids	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_none_defaults_to_standard	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_default	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_missing_added_info	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_with_added_info_missing_id	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_with_added_info_with_id	✅	4ms
engine::validation_non_wasm::tests::test_max_num_bits_verification	✅	4ms
engine::validation_non_wasm::tests::test_select_most_common_dec	✅	4ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_meta_response	✅	6ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_req	✅	5ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_responses	✅	6ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_responses_against_request	✅	7ms
engine::validation_non_wasm::tests::test_validate_request_id	✅	4ms
engine::validation_non_wasm::tests::test_validate_user_decrypt_req	✅	5ms
engine::validation_non_wasm::tests::test_verify_user_decrypt_eip712	✅	5ms
engine::validation_wasm::tests::test_check_ext_user_decryption_signature	✅	7ms
engine::validation_wasm::tests::test_select_most_common_user_dec	✅	4ms
engine::validation_wasm::tests::test_validate_user_decrypt_meta_data_and_signature	✅	6ms
engine::validation_wasm::tests::test_validate_user_decrypt_responses	✅	21ms
engine::validation_wasm::tests::test_validate_user_decrypt_responses_against_request	✅	7ms
grpc::tests::regression_tests::test_request_id_compile_time_interface_stability	✅	4ms
grpc::tests::regression_tests::test_request_id_core_structure_and_api_consistency	✅	4ms
grpc::tests::regression_tests::test_request_id_validation_and_error_handling	✅	4ms
grpc::tests::unit_tests::test_get_meta_store_info_with_real_stores	✅	4ms
grpc::tests::unit_tests::test_get_meta_store_info_with_unavailable_stores	✅	4ms
grpc::tests::unit_tests::test_list_requests_invalid_store_type	✅	4ms
grpc::tests::unit_tests::test_list_requests_pagination	✅	4ms
grpc::tests::unit_tests::test_list_requests_with_real_stores	✅	4ms
grpc::tests::unit_tests::test_list_requests_with_unavailable_stores	✅	4ms
grpc::tests::unit_tests::test_service_with_mixed_store_availability	✅	4ms
testing::material::manager::tests::test_setup_centralized_material	✅	14ms
testing::material::spec::tests::test_centralized_basic_spec	✅	4ms
testing::material::spec::tests::test_comprehensive_spec	✅	4ms
testing::material::spec::tests::test_serialization	✅	4ms
testing::utils::test_purge	✅	7ms
util::file_handling::tests::read_write_element	✅	5ms
util::file_handling::tests::read_write_text	✅	5ms
util::key_setup::tests::test_max_num_bits	✅	3.5s
util::meta_store::tests::auto_remove	✅	5ms
util::meta_store::tests::delete	✅	5ms
util::meta_store::tests::double_insert	✅	4ms
util::meta_store::tests::sunshine	✅	5ms
util::meta_store::tests::test_kickout_of_errors	✅	4ms
util::meta_store::tests::test_subscription	✅	5.0s
util::meta_store::tests::too_many_elements	✅	5ms
util::random_free_port::tests::test_random_free_ports	✅	5ms
util::rate_limiter::tests::test_rate_limiting_1	✅	4ms
util::rate_limiter::tests::test_rate_limiting_more	✅	4ms
util::rate_limiter::tests::test_rate_limiting_refusal	✅	4ms
util::retry::tests::fatal_loop_fails	✅	49ms
util::retry::tests::retry_loop_fails	✅	49ms
util::retry::tests::sunshine_fatal_loop	✅	61ms
util::retry::tests::sunshine_retry_loop	✅	60ms
vault::keychain::secretsharing::tests::test_encrypt_and_decrypt_roundtrip	✅	5ms
vault::keychain::secretsharing::tests::test_new_keychain_without_pub_storage	✅	4ms
vault::keychain::secretsharing::tests::test_operator_public_key_bytes_error	✅	4ms
vault::keychain::secretsharing::tests::test_set_and_get_backup_enc_key	✅	4ms
vault::keychain::tests::test_verify_root_key_measurements	✅	5ms
vault::storage::crypto_material::tests::read_guarded_crypto_material_from_cache_not_found	✅	4ms
vault::storage::crypto_material::tests::read_public_key	✅	5ms
vault::storage::crypto_material::tests::write_central_keys	✅	12ms
vault::storage::crypto_material::tests::write_crs	✅	343ms
vault::storage::file::tests::test_all_data_ids_from_all_epochs_file	✅	7ms
vault::storage::file::tests::test_delete_at_epoch_keeps_dir_when_not_empty	✅	6ms
vault::storage::file::tests::test_delete_at_epoch_removes_empty_epoch_dir	✅	5ms
vault::storage::file::tests::test_epoch_storage	✅	5ms
vault::storage::file::tests::test_overwrite_logic_files	✅	6ms
vault::storage::ram::tests::storage_helper_methods	✅	5ms
vault::storage::ram::tests::test_all_data_ids_from_all_epochs_ram	✅	5ms
vault::storage::ram::tests::test_overwrite_logic_ram	✅	4ms
vault::storage::s3::test_s3_anon	✅	102ms
vault::storage::s3::tests::s3_storage_helper_methods	✅	114ms
vault::storage::s3::tests::test_all_data_ids_from_all_epochs_s3	✅	63ms
vault::storage::s3::tests::test_epoch_methods_in_s3	✅	33ms
vault::storage::s3::tests::test_overwrite_logic_files	✅	21ms
vault::storage::tests::ordered_map	✅	4ms
vault::tests::regression_test_vault_data_type_serialization	✅	4ms
test_backward_compatibility_kms	✅	673ms
test_backward_compatibility_kms_grpc	✅	8ms
kms_custodian_binary_tests::sunshine_decrypt_custodian	✅	158ms
kms_custodian_binary_tests::sunshine_generate	✅	21ms
kms_custodian_binary_tests::sunshine_verify	✅	19ms
kms_gen_keys_binary_test::central_s3	✅	35ms
kms_gen_keys_binary_test::central_signing_address_format	✅	15ms
kms_gen_keys_binary_test::central_signing_keys_overwrite	✅	23ms
kms_gen_keys_binary_test::gen_key_centralized	✅	7.0s
kms_gen_keys_binary_test::gen_key_tempdir_centralized	✅	515ms
kms_gen_keys_binary_test::help	✅	26ms
kms_init_binary_test::help	✅	10ms
kms_init_binary_test::init	✅	14ms
kms_server_binary_test::help	✅	6ms
kms_server_binary_test::subcommand_dev_centralized	✅	5.7s
kms_server_binary_test::test_cert_paths	✅	37ms
client::tests::threshold::user_decryption_tests::default_user_decryption_threshold::case_1::secure_1_true	✅	6m 44s
client::tests::threshold::user_decryption_tests::default_user_decryption_threshold::case_2::secure_1_true	✅	12.9s
client::tests::threshold::user_decryption_tests::default_user_decryption_threshold_precompute_sns::case_1::secure_1_true	✅	12.3s
client::tests::threshold::user_decryption_tests::default_user_decryption_threshold_with_crash::case_1::secure_1_true	✅	12.9s
client::tests::threshold::user_decryption_tests::default_user_decryption_threshold_with_crash::case_1::secure_2_false	✅	12.6s
backup::custodian::tests::internal_custodian_context_duplicate_role_should_fail	✅	5ms
backup::custodian::tests::internal_custodian_context_role_greater_than_nodes_should_fail	✅	5ms
backup::custodian::tests::internal_custodian_context_zero_role_should_fail	✅	5ms
backup::custodian::tests::invalid_threshold_should_fail	✅	4ms
backup::operator::tests::operator_new_fails_with_bad_n_t	✅	4ms
backup::operator::tests::operator_new_fails_with_duplicate_roles	✅	5ms
backup::operator::tests::operator_new_fails_with_insufficient_messages	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_header	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_role	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_timestamp_future	✅	4ms
backup::operator::tests::operator_new_fails_with_invalid_timestamp_past	✅	4ms
backup::operator::tests::operator_new_fails_with_not_enough	✅	4ms
backup::operator::tests::operator_new_fails_with_zero_n	✅	4ms
backup::operator::tests::operator_new_fails_with_zero_t	✅	4ms
backup::operator::tests::operator_timestamp_validation	✅	4ms
backup::operator::tests::validate_recovery_validation_material	✅	5ms
backup::secretsharing::pkcs7::tests::padding_sunshine	✅	4ms
backup::secretsharing::pkcs7::tests::padding_wrong_data	✅	4ms
backup::secretsharing::tests::sharing_missing_shares	✅	89ms
backup::secretsharing::tests::sharing_no_error	✅	142ms
backup::secretsharing::tests::sharing_randomness_test	✅	13.5s
backup::secretsharing::tests::sharing_too_many_missing_shares	✅	11ms
backup::secretsharing::tests::sharing_wrong_params	✅	5ms
backup::secretsharing::tests::sharing_wrong_shares	✅	12ms
backup::seed_phrase::tests::difference	✅	4ms
backup::seed_phrase::tests::mnemonic_robustness	✅	5ms
backup::seed_phrase::tests::sunshine	✅	4ms
backup::tests::custodian_reencrypt	✅	8ms
backup::tests::full_flow::case_1	✅	39ms
backup::tests::full_flow::case_2	✅	26ms
backup::tests::full_flow::case_3	✅	71ms
backup::tests::full_flow_drop_msg	✅	29ms
backup::tests::full_flow_malicious_custodian_init	✅	9ms
backup::tests::full_flow_malicious_custodian_not_enough	✅	11ms
backup::tests::full_flow_malicious_custodian_second	✅	53ms
backup::tests::full_flow_malicious_operator	✅	27ms
backup::tests::operator_setup	✅	7ms
client::crs_gen::tests::process_distributed_crs_result_invalid_signature_does_not_insert_key	✅	8.3s
client::crs_gen::tests::verify_pp_with_tfhers	✅	14.7s
client::tests::centralized::crs_gen_tests::test_crs_gen_centralized	✅	11.0s
client::tests::centralized::crs_gen_tests::test_crs_gen_manual	✅	1.9s
client::tests::centralized::crs_gen_tests::test_insecure_crs_gen_centralized	✅	8.6s
client::tests::centralized::custodian_backup_tests::test_auto_update_backups_central	✅	3.5s
client::tests::centralized::custodian_backup_tests::test_backup_after_crs_central	✅	9.9s
client::tests::centralized::custodian_backup_tests::test_decrypt_after_recovery_central	✅	4.8s
client::tests::centralized::custodian_backup_tests::test_decrypt_after_recovery_centralized_negative	✅	3.5s
client::tests::centralized::custodian_context_tests::test_new_custodian_context_central	✅	3.0s
client::tests::centralized::key_gen_tests::test_decompression_key_gen_centralized	✅	1m 24s
client::tests::centralized::key_gen_tests::test_key_gen_centralized	✅	1.7s
client::tests::centralized::misc_tests_isolated::test_central_close_after_drop_isolated	✅	2.6s
client::tests::centralized::misc_tests_isolated::test_central_health_endpoint_availability_isolated	✅	2.6s
client::tests::centralized::public_decryption_tests::test_decryption_central	✅	1.6s
client::tests::centralized::public_decryption_tests::test_decryption_central_no_decompression	✅	1.6s
client::tests::centralized::public_decryption_tests::test_decryption_central_precompute_sns	✅	1.6s
client::tests::centralized::restore_from_backup_tests_isolated::nightly_test_insecure_central_crs_backup_isolated	✅	3.4s
client::tests::centralized::restore_from_backup_tests_isolated::test_insecure_central_autobackup_after_deletion_isolated	✅	3.2s
client::tests::centralized::restore_from_backup_tests_isolated::test_insecure_central_dkg_backup_isolated	✅	3.2s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized::secure_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized::secure_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_1_true::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_1_true::compression_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_2_false::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns::secure_2_false::compression_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_1_true::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_1_true::compression_2_false	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_2_false::compression_1_true	✅	1.6s
client::tests::centralized::user_decryption_tests::test_user_decryption_centralized_precompute_sns_legacy::secure_2_false::compression_2_false	✅	1.6s
client::tests::common::num_blocks_sunshine	✅	5ms
client::tests::isolated_test_example::test_material_validation	✅	5ms
client::tests::isolated_test_example::tests::test_material_spec_creation	✅	4ms
client::tests::isolated_test_example::tests::test_material_spec_serialization	✅	4ms
client::tests::threshold::crs_gen_tests::test_insecure_crs_gen_threshold	✅	35.7s
client::tests::threshold::custodian_backup_tests::test_auto_update_backups_threshold::case_1	✅	5.1s
client::tests::threshold::custodian_backup_tests::test_auto_update_backups_threshold::case_2	✅	5.1s
client::tests::threshold::custodian_backup_tests::test_backup_after_crs_threshold::case_1	✅	40.4s
client::tests::threshold::custodian_backup_tests::test_backup_after_crs_threshold::case_2	✅	40.1s
client::tests::threshold::custodian_backup_tests::test_decrypt_after_recovery_threshold::case_1	✅	11.2s
client::tests::threshold::custodian_backup_tests::test_decrypt_after_recovery_threshold::case_2	✅	11.1s
client::tests::threshold::custodian_backup_tests::test_decrypt_after_recovery_threshold_negative	✅	4.6s
client::tests::threshold::custodian_context_tests::test_new_custodian_context_threshold::case_1	✅	6.5s
client::tests::threshold::custodian_context_tests::test_new_custodian_context_threshold::case_2	✅	6.5s
client::tests::threshold::key_gen_tests_isolated::test_insecure_dkg_isolated	✅	4.6s
client::tests::threshold::misc_tests::test_threshold_health_endpoint_availability	✅	733ms
client::tests::threshold::misc_tests_isolated::test_threshold_close_after_drop_isolated	✅	517ms
client::tests::threshold::misc_tests_isolated::test_threshold_health_endpoint_availability_isolated	✅	1.5s
client::tests::threshold::misc_tests_isolated::test_threshold_shutdown_isolated	✅	4.4s
client::tests::threshold::mpc_context_tests::test_context_switch_4p	✅	5.0s
client::tests::threshold::public_decryption_tests::test_decryption_threshold::case_1	✅	7.3s
client::tests::threshold::public_decryption_tests::test_decryption_threshold::case_2	✅	7.0s
client::tests::threshold::public_decryption_tests::test_decryption_threshold::case_3	✅	7.0s
client::tests::threshold::public_decryption_tests::test_decryption_threshold_no_decompression::case_1	✅	4.1s
client::tests::threshold::public_decryption_tests::test_decryption_threshold_no_decompression::case_2	✅	3.8s
client::tests::threshold::public_decryption_tests::test_decryption_threshold_no_decompression::case_3	✅	3.8s
client::tests::threshold::public_decryption_tests::test_decryption_threshold_precompute_sns::case_1::compression_1_true	✅	7.0s
client::tests::threshold::public_decryption_tests::test_decryption_threshold_precompute_sns::case_1::compression_2_false	✅	7.0s
client::tests::threshold::restore_from_backup_tests_isolated::nightly_test_insecure_threshold_autobackup_after_deletion_isolated	✅	4.8s
client::tests::threshold::restore_from_backup_tests_isolated::nightly_test_insecure_threshold_dkg_backup_isolated	✅	4.9s
client::tests::threshold::restore_from_backup_tests_isolated::test_insecure_threshold_crs_backup_isolated	✅	13.7s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_1	✅	679ms
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_2	✅	1.4s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_3	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_4	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_5	✅	7.0s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_6	✅	4.1s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold::case_7	✅	4.5s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_all_malicious_failure	✅	2.2s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_malicious::case_1	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_malicious::case_2	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_malicious_failure	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_precompute_sns::case_1	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_precompute_sns::case_2	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_precompute_sns_legacy::case_1	✅	3.8s
client::tests::threshold::user_decryption_tests::test_user_decryption_threshold_precompute_sns_legacy::case_2	✅	3.8s
conf::tests::test_centralized_config	✅	5ms
conf::tests::test_threshold_config	✅	5ms
conf::tests::test_threshold_config_negative	✅	5ms
conf::threshold::test_pem_serialization	✅	5ms
cryptography::decompression::test::test_1024b	✅	3.2s
cryptography::decompression::test::test_128b	✅	1.6s
cryptography::decompression::test::test_16b	✅	1.5s
cryptography::decompression::test::test_2048b	✅	5.1s
cryptography::decompression::test::test_256b	✅	1.8s
cryptography::decompression::test::test_32b	✅	1.5s
cryptography::decompression::test::test_4b	✅	1.5s
cryptography::decompression::test::test_512b	✅	2.3s
cryptography::decompression::test::test_64b	✅	1.5s
cryptography::decompression::test::test_8b	✅	1.5s
cryptography::decompression::test::test_bad_ciphertext	✅	1.2s
cryptography::decompression::test::test_bad_fhe_type	✅	1.5s
cryptography::decompression::test::test_bool	✅	1.5s
cryptography::decompression::test::test_full_chain_client_copro_kms_uint8::case_1	✅	33ms
cryptography::decompression::test::test_full_chain_client_copro_kms_uint8::case_2	✅	1.7s
cryptography::decompression::test::test_tolerate_non_compressed	✅	494ms
cryptography::encryption::tests::nested_pke_sunshine	✅	6ms
cryptography::encryption::tests::pke_wrong_ct_enc	✅	5ms
cryptography::encryption::tests::pke_wrong_kem_key	✅	5ms
cryptography::hybrid_ml_kem::tests::pke_sunshine	✅	39ms
cryptography::hybrid_ml_kem::tests::pke_wrong_ct_hybrid	✅	34ms
cryptography::hybrid_ml_kem::tests::pke_wrong_kem	✅	34ms
cryptography::hybrid_ml_kem::tests::pke_wrong_key	✅	42ms
cryptography::hybrid_ml_kem::tests::pke_wrong_nonce	✅	34ms
cryptography::hybrid_ml_kem::tests::test_pke_serialize_size	✅	5ms
cryptography::hybrid_ml_kem::tests::validate_consistent_cipher_encoding	✅	5ms
cryptography::signatures::tests::bad_dsep	✅	5ms
cryptography::signatures::tests::bad_signature	✅	5ms
cryptography::signatures::tests::plain_signing	✅	5ms
cryptography::signatures::tests::regression_consistent_enc	✅	4ms
cryptography::signatures::tests::sunshine_verf_key_legacy_serialization	✅	4ms
cryptography::signatures::tests::unnormalized_signature	✅	5ms
cryptography::signatures::tests::validate_zeroize_signing_key	✅	4ms
cryptography::signcryption::tests::bad_signcryption	✅	6ms
cryptography::signcryption::tests::incorrect_server_verf_key	✅	4ms
cryptography::signcryption::tests::signcryption_with_bad_link	✅	5ms
cryptography::signcryption::tests::sunshine	✅	5ms
cryptography::signcryption::tests::sunshine_encoding_decoding	✅	6ms
cryptography::signcryption::tests::test_signcryption_payload_v0_serialization_locked	✅	4ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_custodian_role	✅	5ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_operator_role	✅	6ms
engine::backup_operator::tests::test_filter_custodian_data_invalid_signature	✅	5ms
engine::backup_operator::tests::test_filter_custodian_data_missing_verification_key	✅	6ms
engine::backup_operator::tests::test_filter_custodian_missing_cus_output	✅	5ms
engine::base::tests::sunshine_plaintext_as_u256	✅	4ms
engine::base::tests::test_abi_encoding_fhevm_ebytes	✅	4ms
engine::base::tests::test_compute_external_signature_preproc	✅	6ms
engine::base::tests::test_compute_info_crs	✅	673ms
engine::base::tests::test_compute_info_standard_keygen	✅	113ms
engine::base::tests::test_compute_pt_message_hash	✅	5ms
engine::base::tests::test_deserialize_ciphertext_missing_decompression_key	✅	87ms
engine::base::tests::test_deserialize_ciphertext_wrong_ct_format	✅	79ms
engine::base::tests::test_deserialize_ciphertext_wrong_type	✅	79ms
engine::centralized::central_kms::tests::decrypt_with_bad_client_key	✅	182ms
engine::centralized::central_kms::tests::multiple_test_keys_access	✅	28ms
engine::centralized::central_kms::tests::multiple_test_keys_decrypt	✅	44ms
engine::centralized::central_kms::tests::multiple_test_keys_user_decrypt	✅	46ms
engine::centralized::central_kms::tests::sanity_check_sns_compression_test_params	✅	56ms
engine::centralized::central_kms::tests::sunshine_test_decrypt	✅	47ms
engine::centralized::central_kms::tests::sunshine_test_user_decrypt	✅	47ms
engine::centralized::central_kms::tests::test_gen_keys	✅	20.9s
engine::centralized::central_kms::tests::user_decrypt_with_bad_client_key	✅	54ms
engine::centralized::central_kms::tests::user_decrypt_with_bad_ephemeral_key	✅	46ms
engine::centralized::central_kms::tests::user_decrypt_with_bad_sig_key	✅	47ms
engine::centralized::service::crs_gen::tests::already_exists	✅	21.0s
engine::centralized::service::crs_gen::tests::invalid_argument	✅	6ms
engine::centralized::service::crs_gen::tests::not_found	✅	6ms
engine::centralized::service::crs_gen::tests::resource_exhausted	✅	6ms
engine::centralized::service::crs_gen::tests::sunshine	✅	568ms
engine::centralized::service::decryption::test_user_decryption::already_exists	✅	21.1s
engine::centralized::service::decryption::test_user_decryption::invalid_argument	✅	94ms
engine::centralized::service::decryption::test_user_decryption::not_found	✅	93ms
engine::centralized::service::decryption::test_user_decryption::resource_exhausted	✅	91ms
engine::centralized::service::decryption::test_user_decryption::sunshine	✅	95ms
engine::centralized::service::decryption::tests_public_decryption::already_exists	✅	92ms
engine::centralized::service::decryption::tests_public_decryption::invalid_argument	✅	93ms
engine::centralized::service::decryption::tests_public_decryption::not_found	✅	96ms
engine::centralized::service::decryption::tests_public_decryption::resource_exhausted	✅	95ms
engine::centralized::service::decryption::tests_public_decryption::sunshine	✅	97ms
engine::centralized::service::initiator::tests::already_exists	✅	6ms
engine::centralized::service::initiator::tests::invalid_argument	✅	6ms
engine::centralized::service::initiator::tests::sunshine	✅	6ms
engine::centralized::service::key_gen::tests::already_exists	✅	83ms
engine::centralized::service::key_gen::tests::invalid_argument	✅	12ms
engine::centralized::service::key_gen::tests::not_found	✅	86ms
engine::centralized::service::key_gen::tests::resource_exhausted	✅	6ms
engine::centralized::service::key_gen::tests::sunshine	✅	85ms
engine::centralized::service::preprocessing::tests::already_exists	✅	6ms
engine::centralized::service::preprocessing::tests::invalid_argument	✅	6ms
engine::centralized::service::preprocessing::tests::not_found	✅	6ms
engine::centralized::service::preprocessing::tests::resource_exhausted	✅	6ms
engine::centralized::service::preprocessing::tests::sunshine	✅	6ms
engine::context::tests::parse_software_version	✅	4ms
engine::context::tests::test_context_info_duplicate_party_ids	✅	5ms
engine::context::tests::test_software_version_display	✅	4ms
engine::context::tests::test_software_version_equality	✅	4ms
engine::context::tests::test_software_version_major_comparison	✅	4ms
engine::context::tests::test_software_version_minor_comparison	✅	4ms
engine::context::tests::test_software_version_no_tag	✅	4ms
engine::context::tests::test_software_version_patch_comparison	✅	4ms
engine::context::tests::test_software_version_unordered_tag	✅	4ms
engine::context_manager::tests::test_custodian_context	✅	10ms
engine::context_manager::tests::test_gen_recovery_request_payloads	✅	7ms
engine::context_manager::tests::test_kms_context	✅	6ms
engine::context_manager::tests::test_kms_context_load_from_storage	✅	6ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_missing_added_info	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_with_added_info_missing_ids	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_decompression_only_with_added_info_with_ids	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_none_defaults_to_standard	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_default	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_missing_added_info	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_with_added_info_missing_id	✅	4ms
engine::keyset_configuration::tests::test_internal_keyset_config_standard_use_existing_with_added_info_with_id	✅	4ms
engine::threshold::service::crs_generator::tests::already_exists	✅	23.0s
engine::threshold::service::crs_generator::tests::internal_failure	✅	6ms
engine::threshold::service::crs_generator::tests::invalid_argument	✅	5ms
engine::threshold::service::crs_generator::tests::not_found	✅	6ms
engine::threshold::service::crs_generator::tests::resource_exhausted	✅	5ms
engine::threshold::service::crs_generator::tests::sunshine	✅	44ms
engine::threshold::service::crs_generator::tests::unavailable	✅	1m 23s
engine::threshold::service::initiator::tests::already_exists	✅	23.0s
engine::threshold::service::initiator::tests::internal	✅	23.0s
engine::threshold::service::initiator::tests::invalid_argument	✅	5ms
engine::threshold::service::initiator::tests::legacy_prss	✅	6ms
engine::threshold::service::initiator::tests::load_all_prss	✅	5ms
engine::threshold::service::initiator::tests::not_found	✅	5ms
engine::threshold::service::initiator::tests::prss_from_storage_test	✅	5.6s
engine::threshold::service::initiator::tests::sunshine	✅	23.0s
engine::threshold::service::key_generator::tests::aborted	✅	5ms
engine::threshold::service::key_generator::tests::already_exists	✅	35.0s
engine::threshold::service::key_generator::tests::internal	✅	7ms
engine::threshold::service::key_generator::tests::invalid_argument	✅	32.0s
engine::threshold::service::key_generator::tests::not_found	✅	32.0s
engine::threshold::service::key_generator::tests::resource_exhausted	✅	32.0s
engine::threshold::service::key_generator::tests::sunshine	✅	84ms
engine::threshold::service::preprocessor::tests::aborted	✅	23.0s
engine::threshold::service::preprocessor::tests::already_exists	✅	26.0s
engine::threshold::service::preprocessor::tests::internal	✅	6ms
engine::threshold::service::preprocessor::tests::invalid_argument	✅	6ms
engine::threshold::service::preprocessor::tests::not_found	✅	5ms
engine::threshold::service::preprocessor::tests::resource_exhausted	✅	5ms
engine::threshold::service::preprocessor::tests::sunshine	✅	651ms
engine::threshold::service::public_decryptor::tests::already_exists	✅	88ms
engine::threshold::service::public_decryptor::tests::invalid_argument	✅	83ms
engine::threshold::service::public_decryptor::tests::not_found	✅	92ms
engine::threshold::service::public_decryptor::tests::sunshine	✅	87ms
engine::threshold::service::public_decryptor::tests::test_resource_exhausted	✅	83ms
engine::threshold::service::resharer::tests::bad_digests_get_verified_public_materials	✅	18ms
engine::threshold::service::resharer::tests::empty_storage_fetch_public_materials_from_peers	✅	18ms
engine::threshold::service::resharer::tests::sunshine_fetch_public_materials_from_peers	✅	18ms
engine::threshold::service::resharer::tests::sunshine_get_verified_public_materials	✅	11ms
engine::threshold::service::resharer::tests::test_split_devnet_url	✅	4ms
engine::threshold::service::resharer::tests::wrong_digest_fetch_public_materials_from_peers	✅	12ms
engine::threshold::service::user_decryptor::tests::already_exists	✅	85ms
engine::threshold::service::user_decryptor::tests::invalid_argument	✅	85ms
engine::threshold::service::user_decryptor::tests::not_found	✅	89ms
engine::threshold::service::user_decryptor::tests::resource_exhausted	✅	85ms
engine::threshold::service::user_decryptor::tests::sunshine	✅	89ms
engine::validation_non_wasm::tests::test_max_num_bits_verification	✅	5ms
engine::validation_non_wasm::tests::test_select_most_common_dec	✅	5ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_meta_response	✅	6ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_req	✅	5ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_responses	✅	6ms
engine::validation_non_wasm::tests::test_validate_public_decrypt_responses_against_request	✅	7ms
engine::validation_non_wasm::tests::test_validate_request_id	✅	4ms
engine::validation_non_wasm::tests::test_validate_user_decrypt_req	✅	5ms
engine::validation_non_wasm::tests::test_verify_user_decrypt_eip712	✅	5ms
engine::validation_wasm::tests::test_check_ext_user_decryption_signature	✅	7ms
engine::validation_wasm::tests::test_select_most_common_user_dec	✅	4ms
engine::validation_wasm::tests::test_validate_user_decrypt_meta_data_and_signature	✅	6ms
engine::validation_wasm::tests::test_validate_user_decrypt_responses	✅	20ms
engine::validation_wasm::tests::test_validate_user_decrypt_responses_against_request	✅	7ms
grpc::tests::regression_tests::test_request_id_compile_time_interface_stability	✅	4ms
grpc::tests::regression_tests::test_request_id_core_structure_and_api_consistency	✅	4ms
grpc::tests::regression_tests::test_request_id_validation_and_error_handling	✅	4ms
grpc::tests::unit_tests::test_get_meta_store_info_with_real_stores	✅	4ms
grpc::tests::unit_tests::test_get_meta_store_info_with_unavailable_stores	✅	4ms
grpc::tests::unit_tests::test_list_requests_invalid_store_type	✅	5ms
grpc::tests::unit_tests::test_list_requests_pagination	✅	4ms
grpc::tests::unit_tests::test_list_requests_with_real_stores	✅	5ms
grpc::tests::unit_tests::test_list_requests_with_unavailable_stores	✅	4ms
grpc::tests::unit_tests::test_service_with_mixed_store_availability	✅	4ms
testing::material::manager::tests::test_setup_centralized_material	✅	14ms
testing::material::manager::tests::test_setup_threshold_material	✅	23ms
testing::material::spec::tests::test_centralized_basic_spec	✅	4ms
testing::material::spec::tests::test_comprehensive_spec	✅	4ms
testing::material::spec::tests::test_serialization	✅	4ms
testing::material::spec::tests::test_threshold_basic_spec	✅	4ms
testing::utils::test_purge	✅	7ms
util::file_handling::tests::read_write_element	✅	5ms
util::file_handling::tests::read_write_text	✅	4ms
util::key_setup::tests::test_max_num_bits	✅	3.5s
util::meta_store::tests::auto_remove	✅	5ms
util::meta_store::tests::delete	✅	4ms
util::meta_store::tests::double_insert	✅	4ms
util::meta_store::tests::sunshine	✅	4ms
util::meta_store::tests::test_kickout_of_errors	✅	5ms
util::meta_store::tests::test_subscription	✅	5.0s
util::meta_store::tests::too_many_elements	✅	5ms
util::random_free_port::tests::test_random_free_ports	✅	4ms
util::rate_limiter::tests::test_rate_limiting_1	✅	4ms
util::rate_limiter::tests::test_rate_limiting_more	✅	4ms
util::rate_limiter::tests::test_rate_limiting_refusal	✅	4ms
util::retry::tests::fatal_loop_fails	✅	49ms
util::retry::tests::retry_loop_fails	✅	49ms
util::retry::tests::sunshine_fatal_loop	✅	60ms
util::retry::tests::sunshine_retry_loop	✅	60ms
vault::keychain::secretsharing::tests::test_encrypt_and_decrypt_roundtrip	✅	5ms
vault::keychain::secretsharing::tests::test_new_keychain_without_pub_storage	✅	4ms
vault::keychain::secretsharing::tests::test_operator_public_key_bytes_error	✅	5ms
vault::keychain::secretsharing::tests::test_set_and_get_backup_enc_key	✅	4ms
vault::keychain::tests::test_verify_root_key_measurements	✅	4ms
vault::storage::crypto_material::tests::read_guarded_crypto_material_from_cache_not_found	✅	5ms
vault::storage::crypto_material::tests::read_guarded_threshold_fhe_keys_not_found	✅	4ms
vault::storage::crypto_material::tests::read_public_key	✅	4ms
vault::storage::crypto_material::tests::write_central_keys	✅	13ms
vault::storage::crypto_material::tests::write_crs	✅	341ms
vault::storage::crypto_material::tests::write_threshold_empty_update	✅	68ms
vault::storage::crypto_material::tests::write_threshold_keys_failed_storage	✅	62ms
vault::storage::crypto_material::tests::write_threshold_keys_meta_update	✅	64ms
vault::storage::file::tests::storage_helper_methods::threshold_1_true	✅	12ms
vault::storage::file::tests::storage_helper_methods::threshold_2_false	✅	12ms
vault::storage::file::tests::test_all_data_ids_from_all_epochs_file	✅	7ms
vault::storage::file::tests::test_delete_at_epoch_keeps_dir_when_not_empty	✅	6ms
vault::storage::file::tests::test_delete_at_epoch_removes_empty_epoch_dir	✅	5ms
vault::storage::file::tests::test_epoch_storage	✅	6ms
vault::storage::file::tests::test_overwrite_logic_files	✅	6ms
vault::storage::ram::tests::storage_helper_methods	✅	4ms
vault::storage::ram::tests::test_all_data_ids_from_all_epochs_ram	✅	5ms
vault::storage::ram::tests::test_overwrite_logic_ram	✅	5ms
vault::storage::s3::test_s3_anon	✅	103ms
vault::storage::tests::ordered_map	✅	4ms
vault::tests::regression_test_vault_data_type_serialization	✅	4ms
test_threshold_restore_from_backup	✅	1.1s
test_threshold_custodian_backup	✅	1.5s
test_threshold_concurrent_crs	✅	1m 42s
test_threshold_insecure	✅	4m 1s
test_threshold_mpc_context_init	✅	5m 30s
test_threshold_reshare	✅	5m 31s
test_threshold_concurrent_preproc_keygen	✅	7m 29s
test_threshold_mpc_context_switch_6	✅	12m 56s
tests::test_parse_hex	✅	3ms
tests::test_invalid_hex	✅	3ms
tests::test_core_client_config	✅	4ms
crsgen::tests::test_eip712_sigs	✅	448ms
test_backward_compatibility_kms_grpc	✅	3ms
test_backward_compatibility_threshold_fhe	✅	51ms
test_backward_compatibility_kms	✅	627ms
test_centralized_custodian_backup	✅	4.1s
test_centralized_restore_from_backup	✅	4.2s
test_centralized_crsgen_secure	✅	4.3s
test_centralized_insecure	✅	5.4s
identifiers::tests::test_v1_request_id_with_whitespace	✅	3ms
identifiers::tests::test_v1_request_id_to_u128_conversion	✅	3ms
identifiers::tests::request_id_ordering	✅	3ms
identifiers::tests::test_id_type_conversions	✅	3ms
identifiers::tests::test_invalid_id_all_zeros	✅	3ms
identifiers::tests::test_invalid_hex_length	✅	3ms
identifiers::tests::test_v1_request_id_with_prefix	✅	3ms
identifiers::tests::test_key_id_protobuf_conversion	✅	3ms
rpc_types::tests::idempotent_plaintext	✅	3ms
identifiers::tests::test_v1_request_id_conversion	✅	4ms
rpc_types::tests::test_eip712_verification	✅	3ms
rpc_types::tests::test_types_plaintext_ser	✅	3ms
rpc_types::tests::test_enum_default	✅	3ms
identifiers::tests::test_request_id_random	✅	4ms
identifiers::tests::test_valid_hex_characters	✅	3ms
rpc_types::tests::test_request_id_raw_string	✅	3ms
rpc_types::tests::test_old_fhe_type_enum_compatibility	✅	3ms
rpc_types::tests::test_request_id	✅	3ms
rpc_types::tests::test_abi_encoding_fhevm	✅	4ms
identifiers::tests::test_key_id_from_str	✅	4ms
identifiers::tests::test_invalid_v1_request_id	✅	4ms

_{🍂 No flaky tests in this run.}

Github Test Reporter by CTRF 💚

🔄 This comment has been updated

maksymsur · 2025-11-28T12:47:54Z

I temporarily triggered nightly k8s tests in chore(ci): temporary enable k8s tests which will be reverted once I see that these tests successfully executed within this PR

…arty resharing test

…8s test

maksymsur self-assigned this Nov 12, 2025

cla-bot bot added the cla-signed The CLA has been signed. label Nov 12, 2025

maksymsur changed the title ~~refactor(tests): initial setup~~ refactor(tests): adding KMS test material generator with isolated tests infrastructure Nov 12, 2025

maksymsur force-pushed the maksym/refactor/2761/tests-setup branch 3 times, most recently from 4054c6c to 8b35898 Compare November 25, 2025 08:44

maksymsur force-pushed the maksym/refactor/2761/tests-setup branch 14 times, most recently from 9e041b2 to 61e950c Compare November 28, 2025 12:45

maksymsur marked this pull request as ready for review November 28, 2025 12:47

maksymsur requested review from a team as code owners November 28, 2025 12:47

maksymsur marked this pull request as draft November 28, 2025 13:03

maksymsur added 29 commits January 8, 2026 00:40

fix(ci): cert gen and chart ver

97f2f0e

fix(helm): use Helm template unrolling for TLS cert env vars

64b1cf0

fix(helm): configmap and init setup

2e10104

chore(ci): add logs capturing on failed pods

d977711

hore(ci): add logs capturing on failed pods (continue)

16ef372

chore(ci): add logs capturing on failed pods (continue - 2)

1879926

fix(ci): pem annotation

3ca40cf

fix(ci): configmap

032bde4

fix: tests after rebase

266a417

fix: context switch tests after rebase

d2db3fc

fix: context switch tests after rebase - 2

578aa1e

ix: context switch tests after rebase - 3

b1f442d

fix: add config_path and server configs for MPC context creation in p…

d9d9639

…arty resharing test

fix: add config_path and server configs for MPC context creation in K…

be06cc6

…8s test

ci: add early CrashLoopBackOff detection to collect logs faster

6295843

fix(ci): TLS cert upload and improve log collection

0c4e416

fix(ci): proper key setup (save path)

390be90

fix(ci): substitute unsupported aws ci

04c32cb

fix(ci): proper cert parsing

240e06e

chore(ci): run k8s tls test after 6-party setup is fixed

4d2ded2

fix(ci): k8s key fetch endpoint

b4fc50a

chore(ci): refactor k8s tests not to use context switch

60a4ee1

chore: improve k8s threshold tests interface

e0c2896

fix: after merge conflicts

282fffb

fix(ci): test_threshold_mpc_context_init

6c5e985

fix(ci): restore from backup test with epoch id

e43ac8b

fix(ci): test_insecure_central_autobackup_after_deletion_isolated

b28e7f7

fix(ci): ensure_threshold_server_signing_keys_exist

35454a6

fix(ci): test_largecipher_isolated

6cd334d

maksymsur mentioned this pull request Jan 26, 2026

fix: refactor deploy scripts #370

Merged

15 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(tests): adding KMS test material generator with isolated tests infrastructure#270

refactor(tests): adding KMS test material generator with isolated tests infrastructure#270
maksymsur wants to merge 92 commits intomainfrom
maksym/refactor/2761/tests-setup

maksymsur commented Nov 12, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 25, 2025 •

edited

Loading

Uh oh!

maksymsur commented Nov 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

maksymsur commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Description

What Changed

Test Migration - Complete List

1. Core Service Library Tests (core/service/src/client/tests/)

2. CLI Integration Tests (core-client/tests/integration_tests.rs)

Test Migration - Key Generation

File Changes

How It Works

Test Execution Flow

Test Architecture

1. CLI Integration Tests (core-client/tests/integration_tests.rs)

2. Library Tests (core/service/src/client/tests/)

Test Material Generation

Test Material Tool (tools/generate-test-material/)

Makefile Targets

Tool Usage

CI Integration

Material Structure

Testing Infrastructure Architecture

core/service/src/testing/ - Test Infrastructure

CI Configuration

Running Tests

Local Development

Developer Guide

Writing New Tests

Issue ticket number and link

PR Checklist

Dependency Update Questionnaire (only if deps changed or added)

Uh oh!

github-actions bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Consolidated Tests Results 2026-01-07 - 12:57:24

Test Results

Details

test-reporter: Run #2211

❌ Some tests failed!

Tests

Uh oh!

maksymsur commented Nov 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

maksymsur commented Nov 12, 2025 •

edited

Loading

1. Core Service Library Tests (`core/service/src/client/tests/`)

2. CLI Integration Tests (`core-client/tests/integration_tests.rs`)

1. CLI Integration Tests (`core-client/tests/integration_tests.rs`)

2. Library Tests (`core/service/src/client/tests/`)

Test Material Tool (`tools/generate-test-material/`)

`core/service/src/testing/` - Test Infrastructure

github-actions bot commented Nov 25, 2025 •

edited

Loading