feat(FR-2877): implement webui-smoke-cli runner with tag filter and external endpoint

[yomybaby]

Resolves #7381(FR-2877)

Part of Epic FR-2871 — WebUI Smoke CLI. Stacked on top of #7391.

Summary

• Implement bai-smoke run against external WebUI + webserver endpoints
• Role auto-detection from /server/login response (defaults to user on detection failure)
• Tag / page filters (--include, --exclude, --pages); grep regex composed as (@smoke|@smoke-any|@smoke-{role})
• Spawn Playwright with the bundled playwright.smoke.config.ts that extends repo e2e/ directory
• Playwright HTML + JSON reports + a small summary.json (env + role + counts)
• Resolve WEBUI_SHA (via git rev-parse HEAD fallback) and PLAYWRIGHT_VERSION (from @playwright/test/package.json)
• Add @playwright/test as a direct dep on the package

Out of scope (Phase 2)

• doctor / preflight subcommands → FR-2878
• Rich diagnostic block in report (console, network, traces grouping) → FR-2879
• Chromium bundling / air-gap / TLS bundling → FR-2881
• Single executable binary → FR-2882
• Operator-facing README EN+KO → FR-2883

Verification

• pnpm install + pnpm --filter backend.ai-webui-smoke-cli build → PASS
• bai-smoke version / bai-smoke list / bai-smoke run --help → PASS, all new flags documented
• bash scripts/verify.sh → Relay / Lint / Format PASS. TypeScript fails only on pre-existing errors on unrelated files — same set already present on main and PR feat(FR-2875,FR-2876): introduce @smoke tag convention and scaffold webui-smoke-cli package #7391.
• tsc -p packages/backend.ai-webui-smoke-cli/tsconfig.json --noEmit → clean.

Live staging run (added post-implementation)

Ran against staging webserver https://3398ac.public.isla-sorna.backend.ai (same host serves WebUI):

bai-smoke run \
  --endpoint https://3398ac.public.isla-sorna.backend.ai \
  --webserver https://3398ac.public.isla-sorna.backend.ai \
  --email admin@lablup.com --password '***' \
  --role auto --output /tmp/smoke-mvp-test \
  --timeout 120s --workers 1 --include @smoke-any

Runner-level acceptance: PASS

• Playwright spawned, 39 tagged specs discovered and run, full lifecycle in ~3 min
• Role auto-detection: webserver login → resolved admin
• summary.json written with env + role + counts (total: 39, passed: 0, failed: 29, skipped: 10)
• HTML + JSON reports written to output dir
• Exit code propagated correctly (1, since some tests failed)

Spec-level result: 29 failures, all sharing the same root cause — e2e/utils/test-util.ts:136 cannot find the "Password" label on the login page of this staging instance. This is not a smoke-CLI bug; it is an upstream e2e helper / staging-UI mismatch that already affects regular e2e runs against this server. Tracked as a separate concern, not blocking this PR.

Observation flagged for review — --include @smoke-any produced the composed grep (@smoke\b|@smoke-any\b|@smoke-admin\b|@smoke-any). The user-supplied include is OR'd with the implicit @smoke base, so --include cannot currently narrow the run; it can only widen it. Decide in review whether to:

1. Keep current behavior (additive only — document it).
2. Switch to strict intersection when --include is given.
   The current behavior is acceptable for MVP but worth resolving before Phase 2.

TODO markers left for Phase 2

• WEBUI_SHA build-time injection is still best-effort via git rev-parse; FR-2881 will replace this with a generated module shipped in the tarball.

🤖 Generated with Claude Code

[yomybaby]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(FR-2877): implement webui-smoke-cli runner with tag filter and external endpoint #7392 👈 (View in Graphite)
• feat(FR-2875,FR-2876): introduce @smoke tag convention and scaffold webui-smoke-cli package #7391
• chore(FR-2872): add WebUI Smoke CLI dev plan #7389
• feat(FR-2872): add WebUI Smoke CLI feature spec #7375
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• flow:merge-queue - adds this PR to the back of the merge queue
• flow:hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Pull request overview

Implements the FR-2877 MVP “smoke CLI runner” by wiring bai-smoke run to spawn Playwright against an operator-supplied WebUI/webserver endpoint, applying smoke-tag filtering and emitting HTML/JSON reports plus a lightweight summary.json.

Changes:

• Add a Playwright runner (runSmoke) that composes env/grep filters, spawns Playwright with a smoke-specific config, and writes summary.json.
• Introduce CLI option parsing for run (endpoint/webserver/auth/role/pages/tag filters/workers/timeout/output) and add password-stdin + argv scrubbing.
• Add a smoke Playwright config that reuses the repo’s e2e/ tree and produces HTML + JSON reports; add @playwright/test as a direct dependency and expose Playwright/WebUI SHA in version.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/backend.ai-webui-smoke-cli/src/version.ts	Resolve CLI version, Playwright version, and best-effort WebUI SHA.
packages/backend.ai-webui-smoke-cli/src/runner.ts	Spawn Playwright with smoke config, auto-detect role, and generate summary.json.
packages/backend.ai-webui-smoke-cli/src/config.ts	Pure helpers to map CLI options → env vars + grep/grepInvert regex.
packages/backend.ai-webui-smoke-cli/src/cli.ts	Implement bai-smoke run command and option parsing (incl. password-stdin).
packages/backend.ai-webui-smoke-cli/README.md	Document alpha usage, flags, and MVP limitations.
packages/backend.ai-webui-smoke-cli/playwright.smoke.config.ts	Smoke-focused Playwright config pointing at repo e2e/ with report outputs.
packages/backend.ai-webui-smoke-cli/package.json	Ship smoke config in package files and add @playwright/test dependency.