Commit e446c37
authored
fix(test-suite): add resilient smoke runner for inputFlow (release) (#1921)
fix(test-suite): add resilient smoke runner for inputFlow (#1840)
* fix(test-suite): add smoke runner hardening
* fix(test-suite): document smoke env setup
* fix(test-suite): simplify env handling + post-success cleanup
- Simplify RPC URL warning conditions (redundant checks removed)
- Refactor cancelBacklog to reuse sendWithRetries for resilience
- Add post-success cleanup to clear backlogs on unclean signers
* style: apply prettier formatting
* feat(test-suite): add BetterStack heartbeat integration
- Ping BETTERSTACK_HEARTBEAT_URL on success
- Report failure status with error message
- Document in README
* fix(test-suite): use receipt.contractAddress instead of computing it
More reliable: reads the actual deployed address from the receipt
rather than computing it from nonce, eliminating potential mismatch.
* fix(test-suite): doc and deprecated API cleanup
- README: correct SMOKE_CANCEL_BACKLOG default from '1' to 'true'
- smoke-inputflow: remove deprecated provider.getGasPrice() call,
use MIN_PRIORITY_FEE as last-resort fallback instead
* fix(test-suite): remove unnecessary hardhat chainId check and redundant casts
- E2ECoprocessorConfigLocal.sol: remove 31337 (hardhat) from chainId check,
smoke tests target real networks only
- instance.ts: remove redundant Number() casts, values are already numbers
after Number.isFinite() validation
* fix(test-suite): add robustness for mainnet API key, decryption timeouts, and gas estimation
- instance.ts: enforce ZAMA_FHEVM_API_KEY on mainnet to fail fast
- smoke-inputflow.ts: add withTimeout wrapper for decryption operations
(default 120s, configurable via SMOKE_DECRYPT_TIMEOUT_SECS)
- smoke-inputflow.ts: wrap gas estimation in try/catch with clear error messages
- README.md: document SMOKE_DECRYPT_TIMEOUT_SECS env var
* fix(test-suite): log full error on gas estimation failure for debugging
Add console.error logging before re-throwing in gas estimation catch
blocks to preserve full error details (stack trace, ethers properties)
for debugging purposes.
* refactor(test-suite): delegate coprocessor config entirely to ZamaConfig
ZamaConfig.getEthereumCoprocessorConfig() already handles chainId checks
internally for mainnet (1), Sepolia (11155111), and local (31337),
returning appropriate addresses for each. Remove redundant chainId check
and hardcoded fallback addresses.
* refactor(test-suite): simplify RPC URL config and require network-specific URLs
- staging/zwsDev: use RPC_URL with localhost fallback
- sepolia: require SEPOLIA_ETH_RPC_URL explicitly (fail if not set)
- mainnet: require MAINNET_ETH_RPC_URL explicitly (fail if not set)
Removes verbose RPC_URL fallback chains - if targeting a specific
network, set its specific env var. Clearer and less error-prone.
* refactor(test-suite): simplify instance.ts config - remove unnecessary vars
Only ZAMA_FHEVM_API_KEY is a secret that benefits from hardhat vars.
Contract addresses, chain IDs, and URLs can use simple env || defaults.
Removes verbose chaining and string↔number conversions.
* refactor(test-suite): use BetterStack /fail endpoint for failure reporting
Simpler than encoding error messages in query params. Error details
are already logged to console.
* refactor(test-suite): simplify env var parsing in smoke runner
- Remove unnecessary Math.trunc() calls
- Use simple boolean coercion for allowCancel and forceDeploy
- Drop intermediate variables (cancelRaw, forceRaw)
* refactor(test-suite): address PR feedback for smoke runner
- Rename SMOKE_FORCE_DEPLOY → SMOKE_DEPLOY_CONTRACT (defaults to 1)
- Add SMOKE_RUN_TESTS to allow deploy-only mode (set to 0)
- Add strict boolean parsing for env vars (only 0/1, fail fast on invalid)
- Send error details to BetterStack /1 endpoint instead of /fail
* feat(test-suite): add timing breakdown to smoke heartbeat
- Track deploy, encrypt, tx, and decrypt phases separately
- Send timing report to BetterStack on success
- Align timing boundaries with TransferBench (decrypt excludes handle fetch)
* fix(test-suite): lazy validation for network RPC URLs
Only throw error for missing SEPOLIA_ETH_RPC_URL / MAINNET_ETH_RPC_URL
when actually targeting that network, not during compile/build.
Fixes Docker build failure.
* feat(test-suite): add RPC_URL fallback for sepolia/mainnet
For pod deployments, just set RPC_URL - it works for all networks.
Keeps network-specific vars (SEPOLIA_ETH_RPC_URL, MAINNET_ETH_RPC_URL)
for local dev convenience.
* docs(test-suite): clarify env setup for sepolia/mainnet/devnet
- sepolia/mainnet: SDK provides defaults, only need RPC_URL + MNEMONIC
- devnet: use pre-configured .env.devnet
- other networks: set all vars manually
* fix(test-suite): provide placeholder URLs for sepolia/mainnet during compile
When not targeting sepolia/mainnet, provide placeholder public RPC URLs
to satisfy Hardhat's config validation. The actual RPC URL is still
required when targeting these networks.
* refactor(test-suite): move TestInput.sol to contracts/smoke/
Allows faster container startup with:
npx hardhat compile:specific --contract contracts/smoke
* feat(test-suite): improve smoke signer handling and observability
- Default to 3 signers (0,1,2) for automatic failover
- Dynamic balance threshold based on current gas prices
- Show all available signers at startup with balances
- Add LOW_BALANCE warning for signers < 0.1 ETH
- Fix withTimeout to clear timer on success (prevents zombie timers)
- Update README with signer funding guidance
* fix(test-suite): log warning when failure heartbeat fails
Add consistent warning behavior for the failure heartbeat to match
the success heartbeat logging.
* revert(test-suite): restore hardcoded addresses in E2ECoprocessorConfigLocal
The ZamaConfig approach doesn't work for devnet/staging which uses
different contract addresses than production Sepolia despite sharing
the same chainId.
* feat(test-suite): inject coprocessor config via constructor for smoke tests
- TestInput.sol now accepts (acl, coprocessor, kmsVerifier) in constructor
- instance.ts exports addresses with SDK defaults for sepolia/mainnet
- smoke-inputflow.ts passes addresses when deploying
This allows smoke tests to work on:
- Sepolia/mainnet: uses SDK defaults
- Devnet: uses env vars (FHEVM_EXECUTOR_CONTRACT_ADDRESS, etc.)
Other E2E tests still use sed-patched E2ECoprocessorConfigLocal.sol.
* refactor(test-suite): split SmokeTestInput from TestInput to avoid E2E regression
- Create SmokeTestInput.sol with constructor-based config injection for smoke tests
- Revert TestInput.sol to use E2ECoprocessorConfig inheritance for Pierre's E2E tests
- Update smoke-inputflow.ts to use SmokeTestInput with proper TypeScript typing
This avoids breaking existing E2E tests (inputFlow.ts, pausedHost.ts, pausedGateway.ts)
that deploy TestInput without constructor arguments.
* fix(test-suite): handle late tx confirmation in retry loop
When a tx times out and we attempt to send a replacement, the original tx
might get mined in that window. This caused false failure reports because
the replacement send would fail with "nonce already used" and we'd exhaust
retries without recognizing the original tx succeeded.
Fix: Track all sent tx hashes and check for late receipts when:
- A send fails (might be because previous tx just got mined)
- After exhausting all retries (final safety check)
* refactor(test-suite): use tx.wait() with TRANSACTION_REPLACED handling
Replace manual polling (waitForReceipt) and hash tracking (sentTxHashes)
with ethers v6 native tx.wait() which handles replacement detection via
TRANSACTION_REPLACED error code.
This simplifies the retry logic:
- Remove waitForReceipt function (ethers polls internally)
- Remove sentTxHashes tracking (ethers tracks replacements)
- Remove checkPreviousTxs helper (ethers throws TRANSACTION_REPLACED)
- Catch TRANSACTION_REPLACED and return replacement receipt if successful
Co-authored-by: maxnovawind
* fix(test-suite): handle ethers v6 tx.wait() errors correctly
- Handle CALL_EXCEPTION explicitly (reverts are terminal, don't retry)
- Handle TIMEOUT explicitly (retry with bumped fees)
- Refresh signer states before cleanup to avoid stale nonce issues
* fix(test-suite): add retry logic for hardhat compile in Docker build
* Revert "fix(test-suite): add retry logic for hardhat compile in Docker build"
This reverts commit 452d29c.
* fix(test-suite): make API key authentication optional
Internal smoke tests don't go through Kong, so the API key
requirement on mainnet was blocking internal testing workflows.
* fix(test-suite): use network-suggested priority fee instead of 2 gwei floor
Remove the hardcoded 2 gwei minimum priority fee and use the network's
recommended value instead. Falls back to 0.1 gwei only if network returns null.
This reduces priority fees from ~2 gwei to ~0.0001 gwei on mainnet,
significantly lowering transaction costs during normal network conditions.
* fix(test-suite): improve smoke logging and gas caps
* fix(test-suite): address PR review feedback
- Fix timer leak in withTimeout by using .finally() instead of .then()
- Remove unused extendProvider import from hardhat.config.ts
- Fetch fresh balance after backlog cancellation to fix stale log output
- Upgrade heartbeat and cleanup failure logs from warn to error
* fix(test-suite): responder-friendly smoke failure payload1 parent 98c0881 commit e446c37
File tree
11 files changed
+1210
-52
lines changed- test-suite/e2e
- contracts
- smoke
- scripts
- test
11 files changed
+1210
-52
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
| 36 | + | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
2 | 7 | | |
3 | 8 | | |
4 | 9 | | |
| |||
9 | 14 | | |
10 | 15 | | |
11 | 16 | | |
| 17 | + | |
12 | 18 | | |
13 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
7 | 9 | | |
8 | 10 | | |
9 | 11 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
| 3 | + | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
27 | 26 | | |
28 | 27 | | |
29 | 28 | | |
| |||
90 | 89 | | |
91 | 90 | | |
92 | 91 | | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
93 | 100 | | |
94 | 101 | | |
95 | 102 | | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | 103 | | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
109 | 107 | | |
110 | 108 | | |
111 | 109 | | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
117 | 116 | | |
118 | 117 | | |
119 | 118 | | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
| |||
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
165 | | - | |
| 165 | + | |
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
| |||
0 commit comments