feat(cli): add --scenario flag with name matching and list command#13
Merged
Conversation
Summary: - Add --scenario flag as alias for --scenario-id with name matching support - Support comma-separated scenario IDs/names in --scenario flag - Match scenarios by both id and name (e.g., --scenario "Refund smoke question") - Improve error message to list available scenarios with IDs and names - Add list command to display available scenarios for discovery - Extend selectDashboardScenarios with same name-matching logic - Fix db.test.ts expected error message to match new format Rationale: - Users want to run a single scenario without running the entire file. - Matching by name is more ergonomic than requiring exact kebab-case IDs. - The list command helps users discover scenario IDs/names before running. - Comma-separated values allow running multiple specific scenarios. Tests: - bun run test: 82 pass, 0 fail - bun run test:e2e: 15 pass, 0 fail - Added unit test for scenarioId filtering by name in runSuite - Added unit test for comma-separated scenario selection - Added e2e test for --scenario flag alias - Added e2e test for name-based scenario matching Co-authored-by: Codex <codex@openai.com>
Summary: - Update scenario filters spec to describe --scenario flag, name matching, comma-separated values, and improved error messages - Add "List command shows available scenarios" spec section - Add list command to current-state.md as implemented - Add list command to e2e-checklist.md as planned - Add e2e test for no-match scenario-id returning available IDs Rationale: - Product spec must reflect the --scenario and list command behavior introduced in the previous commit before marking work complete. Tests: - bun run fast-feedback: all gates pass (repo validation, lint, typecheck, 85 tests) Co-authored-by: Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Add support for running specific scenarios by ID or name, and a
listcommand for scenario discovery. Closes SYM-16.Previously,
--scenario-idonly accepted a single exact ID match. Users had no way to run a scenario by its human-readable name, discover available scenarios, or select multiple scenarios at once.Behavior changes
--scenario <id-or-name>flag added as an alias for--scenario-id, matching by both scenario ID and scenario name--scenario id1,name2runs multiple scenarioslistcommand:agentprobe list --scenarios <path>prints each scenario's ID, name, and tagsValidation
./scripts/fast-feedback.shpassed (repo validation, lint, typecheck, 85 tests)Screenshots / video
N/A for CLI-only changes.