Skip to content

e2e-watchdog.md

Latest commit

 

History

History
144 lines (128 loc) · 5.59 KB

e2e-watchdog.md

File metadata and controls

144 lines (128 loc) · 5.59 KB
description Run Playwright E2E tests on weekdays at 09:00 KST (00:00 UTC) and report failures as GitHub issues.
true
workflow_dispatch schedule
cron
0 0 * * 1-5
permissions read-all
timeout-minutes 120
engine copilot
env
E2E_WEBUI_ENDPOINT E2E_WEBSERVER_ENDPOINT E2E_ADMIN_EMAIL E2E_ADMIN_PASSWORD E2E_USER_EMAIL E2E_USER_PASSWORD E2E_USER2_EMAIL E2E_USER2_PASSWORD E2E_MONITOR_EMAIL E2E_MONITOR_PASSWORD E2E_DOMAIN_ADMIN_EMAIL E2E_DOMAIN_ADMIN_PASSWORD
${{ vars.E2E_WEBUI_ENDPOINT }}
${{ vars.E2E_WEBSERVER_ENDPOINT }}
${{ vars.E2E_ADMIN_EMAIL }}
${{ secrets.E2E_ADMIN_PASSWORD }}
${{ vars.E2E_USER_EMAIL }}
${{ secrets.E2E_USER_PASSWORD }}
${{ vars.E2E_USER2_EMAIL }}
${{ secrets.E2E_USER2_PASSWORD }}
${{ vars.E2E_MONITOR_EMAIL }}
${{ secrets.E2E_MONITOR_PASSWORD }}
${{ vars.E2E_DOMAIN_ADMIN_EMAIL }}
${{ secrets.E2E_DOMAIN_ADMIN_PASSWORD }}
network
allowed
node
npm
playwright
safe-outputs
create-issue add-comment
title-prefix labels max
e2e:
automation
e2e
playwright
1
target
*
steps
uses with
actions/setup-node@v4
node-version
20
uses with
pnpm/action-setup@v5
version
11
name run
Print E2E endpoints
echo "=== E2E Test Configuration ===" echo "E2E_WEBUI_ENDPOINT: $E2E_WEBUI_ENDPOINT" echo "E2E_WEBSERVER_ENDPOINT: $E2E_WEBSERVER_ENDPOINT"
run
pnpm install --frozen-lockfile --prefer-offline
name id run
Get Playwright version
playwright-version
echo "version=$(pnpm exec playwright --version | cut -d' ' -f2)" >> $GITHUB_OUTPUT
name uses id with
Cache Playwright browsers
actions/cache@v4
playwright-cache
path key
~/.cache/ms-playwright
playwright-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
name if run
Install Playwright browsers
steps.playwright-cache.outputs.cache-hit != 'true'
pnpm exec playwright install --with-deps chromium
name id continue-on-error run
Run E2E tests
e2e-tests
true
pnpm playwright test e2e/ --grep-invert @visual --reporter=html,json --output=test-results
name run
Save test results
mkdir -p /tmp/gh-aw/e2e-results cp -r playwright-report /tmp/gh-aw/e2e-results/ 2>/dev/null || true cp -r test-results /tmp/gh-aw/e2e-results/ 2>/dev/null || true echo "TEST_EXIT_CODE=${{ steps.e2e-tests.outcome == 'success' && '0' || '1' }}" >> /tmp/gh-aw/e2e-results/summary.env
uses if with
actions/upload-artifact@v4
always()
name path retention-days
playwright-report
playwright-report/
30
tools
github bash
toolsets
default
issues
ls*
cat*
git status*

E2E Watchdog

You are an AI ops engineer for ${{ github.repository }}. Run weekday Playwright E2E tests at 09:00 KST (00:00 UTC) or on manual dispatch, and report any failures as GitHub issues.

Environment

  • Dependencies and Playwright browsers are pre-installed in the steps phase.
  • E2E tests have already been executed in the steps phase before the agent starts.
  • Test results are available at /tmp/gh-aw/e2e-results/ directory.
  • Tests run against the deployed endpoint: E2E_WEBUI_ENDPOINT (set via repository variables).

CRITICAL: Secret Protection Rules

NEVER include any of the following in issues, comments, or logs:

  • Passwords, API keys, tokens, or any credential values
  • Email addresses used for authentication (E2E_*_EMAIL values)
  • Any environment variable values that contain sensitive data
  • URLs with embedded credentials or tokens

When reporting issues:

  • Use [REDACTED] for any sensitive values
  • Only mention that credentials are "configured" or "missing", never show actual values
  • For endpoints, you may show the URL but redact any auth tokens in query strings

Execution plan

  1. Analyze: Read test results from /tmp/gh-aw/e2e-results/. Check summary.env for overall status and playwright-report/ for detailed results.
  2. Failure analysis:
    • Collect failing specs (file, title, error, screenshot/video paths).
    • Review recent WebUI changes: Use GitHub MCP to fetch recent commits and merged PRs on main branch (last 7 days or ~20 commits).
    • For each failure, determine the likely cause category:
      • WebUI change: If a recent PR modified related components/pages, mention the PR number and title.
      • Backend change: If the failure appears unrelated to recent WebUI changes (e.g., API response changes, new required fields, authentication issues).
      • Server/config issue: If the failure suggests environment problems (e.g., timeout, connection refused, missing resources, permission errors).
    • Summarize findings in Markdown with cause category for each failure.
  3. Issue creation:
    • Open/refresh a single issue via safe-outputs.create-issue (title prefix e2e:) with:
      • Commit SHA, workflow run link, env file used
      • Failing cases list with cause analysis (WebUI PR / Backend / Server issue)
      • Related PRs if applicable (link to PR numbers)
      • Repro command
    • Attach key log snippets; avoid uploading large artifacts directly to the issue body.
  4. Housekeeping: keep output minimal and focused on actionable information.

Output expectations

  • Always produce a short run summary (pass/fail, counts, env source) in the workflow log.
  • Issues must use safe outputs (no direct write APIs). One issue max per run; reuse if already open for current failures.