refactor(test-suite): replace fhevm bash cli with lean Bun TS#2134
Open
refactor(test-suite): replace fhevm bash cli with lean Bun TS#2134
Conversation
Changed Lines CoverageCoverage of added/modified lines: N/A |
Eikix
commented
Mar 20, 2026
Contributor
Author
Eikix
left a comment
Eikix
left a comment
There was a problem hiding this comment.
[AUTOMATED]
Code Review Summary
Reviewed for bugs, guideline compliance, and error handling across all 39 changed files.
Findings
| Category | Count | Severity |
|---|---|---|
| Bug | 1 | High — CI correctness |
| Naming (guideline) | 3 | Nit |
| Error handling | 3 | Medium |
The BUILD env var issue in the e2e workflow is the most impactful finding — it causes unnecessary image rebuilds in the orchestrated CI path. The naming and error handling items are non-blocking improvement suggestions.
Overall the PR is well-structured: the compose path migration is consistent, the CI consolidation is clean, and the documentation is thorough.
Eikix
force-pushed
the
elias/fhevm-cli-clean
branch
from
9dd6023 to
a2e4ba6
Compare
Eikix
commented
Mar 25, 2026
Eikix
commented
Mar 25, 2026
Eikix
commented
Mar 25, 2026
Eikix
commented
Mar 25, 2026
Contributor
Author
Eikix
left a comment
Eikix
left a comment
There was a problem hiding this comment.
[AUTOMATED]
Code Review
Reviewed for bugs, security issues, guideline compliance, and error handling.
Result: 1 issue found (error handling). No bugs or security issues detected.
The CLI is well-structured with thorough input validation, proper async/await patterns, correct shell escaping for sh -lc commands, and a solid typed error hierarchy. The 15-step boot pipeline, state persistence, and resume logic all look correct.
Eikix
commented
Mar 25, 2026
Eikix
commented
Mar 25, 2026
Contributor
Author
|
Mob review comment: print each version of the stack at start |
PanGan21
previously approved these changes
Mar 26, 2026
Eikix
added
the
e2e orchestrate test
PanGan21
previously approved these changes
Mar 26, 2026
* feat(test-suite): add multi-chain with option for mutli-copro e2e testing support * feat(test-suite): add multi-chain host topology with imrpoved orchestration * refactor(test-suite): address PR review, harden multi-chain topology model * refactor(test-suite): centralize primary host key and eliminate ad-hoc string * fix(test-suite): use per-chain contract addresses in multi-chain tests * refactor(test-suite): make chain keys slug-native and remove naming/API smells
Pause test profiles (paused-host-contracts, paused-gateway-contracts) now handle their own pause/unpause wrapping inside runProfile instead of relying on runStandardSuite to set up contract state. This makes standalone `fhevm-cli test paused-gateway-contracts` work correctly instead of silently running against an unpaused stack.
Adds label-based trigger so the full orchestrate pipeline can be exercised on a regular PR before merge queue.
Docker Desktop for Mac has a race condition where `docker compose up -d` can create a network, assign it to a container, then find the network gone by the time the container starts. This happens especially during rapid teardown/recreate cycles. Fix: pre-create the project network with correct compose labels before the first `composeUp` call. Also explicitly sweep stale networks in `down()` after component teardown, matching the cleanup the no-state path already does.
…chdog Instead of blanking GATEWAY_RPC_URL to disable the ConsensusWatchdog during the ciphertext-drift test, pass EXPECT_CIPHERTEXT_DIVERGENCE=true so both detection layers run and confirm the injected divergence. Also fix the orchestrate label trigger to match the actual GitHub label.
…ft test After the gw-listener confirms the drift warning, query the gateway chain for AddCiphertextMaterial events and verify the two coprocessor submissions produced distinct ciphertext digests. Reverts the earlier ConsensusWatchdog changes — the watchdog is a safety net for normal tests, not the right place for intentional drift assertions.
The resolve step during normal boot only printed the lock name (e.g. latest-main-73e9e30.json) without the individual component versions. The dry-run path already printed the detailed breakdown. Now both paths print the full version matrix so CI logs always show exactly which image tags were used.
Eikix
force-pushed
the
elias/fhevm-cli-clean
branch
from
d146576 to
a625d31
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes https://github.com/zama-ai/fhevm-internal/issues/1042
Supersedes #2066.
#2066keeps the iteration history and design exploration. This PR is the same final direction with a clean atomic history for review.Summary
Replace the legacy
test-suite/fhevm/fhevm-clibash flow with a lean Bun/TypeScript CLI.The new CLI makes target resolution, runtime generation, scenarios, local overrides, resume, upgrade, and test execution explicit instead of spreading that behavior across bash scripts and CI glue.
What changed
test-suite/fhevm/srcdocker-compose/templates/static/profiles/scenarios/.fhevm/with a hard split between:.fhevm/state.fhevm/runtimelatest-supportedlatest-mainsha --sha <git-sha>devnettestnetmainnetdeployas an alias forupdoctorwithup --dry-runCI contract
latest-main --buildwith the checked-intwo-of-twoscenario./fhevm-cli test lightsuite instead of copying those steps in YAMLbuild=falsebun run checkbun test srcValidation
Verified locally on this branch:
cd test-suite/fhevm && bun run checkcd test-suite/fhevm && bun test srcNotes
latest-supportedis now a tracked maintained profilelatest-mainis the moving baseline for local and PR validationtwo-of-twois the canonical checked-in multicopro scenario preset