Skip to content

Add CLI error-scenario smoke test matrix for devs #470

Description

@SgtPooki

Description

Manually verifying CLI error scenarios across all commands is slow and ad-hoc. During PR #468 (CliFatal sentinel work) we hand-rolled a 15-scenario bash sweep to confirm error output and exit codes. We should turn this into a reusable script or test suite so future CLI changes can re-run the matrix in one command.

Goals

  • Cover every command + every documented error scenario (invalid args, missing flags, bad RPC, no auth, insufficient funds, user cancel, etc.)
  • Capture stdout, stderr, exit code per scenario
  • Run without live RPC where possible (use mocks or no-network failure paths)
  • Clear pass/fail per scenario; easy to add new scenarios

Acceptance criteria

  • pnpm run smoke (or similar) executes the full matrix
  • Output format makes it easy to spot regressions (snapshot diff or labeled per-scenario block)
  • At minimum covers: payments setup/fund/deposit/withdraw/status, add, import, data-set show/list/terminate, provider list/show/ping, rm
  • Argument-validation scenarios (no network, no auth) run in CI
  • Network-dependent scenarios (insufficient funds, bad RPC) are clearly tagged and skipped in CI by default
  • README or CONTRIBUTING.md documents how to run + add scenarios

Notes

  • See PR refactor: finish CLI command cleanup #468 conversation for the manual sweep template
  • Reuse existing src/test/mocks/synapse-sdk.js where mocking is preferable to live RPC
  • Consider snapshot testing on captured stderr (vitest toMatchInlineSnapshot)
  • Out of scope: replacing existing unit/integration tests

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestgood first issueGood for newcomershelp wantedExtra attention is neededready-for-workTriaged: scope, plan, and DoD are clear; contributor can pick upteam/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions