|
| 1 | +# Mandatory QA merge gate |
| 2 | + |
| 3 | +Makes feature verification a **required, mechanical** check before a PR can |
| 4 | +merge to `main` — closing the class of gap that let MCP-1214 ship (a native |
| 5 | +macOS tray bug that Web-UI-only QA never exercised). |
| 6 | + |
| 7 | +The `gh pr merge --admin` escape hatch is intentionally retained |
| 8 | +(`enforce_admins` stays `false`) for genuine emergencies; normal merges must be |
| 9 | +green. |
| 10 | + |
| 11 | +## Required checks |
| 12 | + |
| 13 | +| Check (status context) | Where it runs | Catches | |
| 14 | +|---|---|---| |
| 15 | +| `swift-test` | `.github/workflows/native-tests.yml` (macOS) | Native tray form logic — validation, dirty detection, tri-state durations. GUI-free, deterministic. | |
| 16 | +| `settings-parity` | `.github/workflows/native-tests.yml` (Linux) | Web (`fields.ts`) ↔ native (`SettingsCatalog.swift`) duration-field drift (placeholder / optional). | |
| 17 | +| `qa-gate` | Paperclip QATester → `scripts/post-qa-gate-status.sh` | Full feature QA. `success` only when QATester PASSes at the PR's **current head SHA**. | |
| 18 | + |
| 19 | +`swift-test` and `settings-parity` are ordinary GitHub Actions jobs, but |
| 20 | +`native-tests.yml` is deliberately **required-safe**: the workflow has no |
| 21 | +top-level `paths:` filter, so it runs on *every* PR. A `changes` job |
| 22 | +(`dorny/paths-filter`) detects whether native / settings files were touched, |
| 23 | +and the two real jobs are gated with a job-level `if:`. On a PR that touches |
| 24 | +none of those paths the jobs are **skipped**, and a skipped job reports its |
| 25 | +required context as satisfied (green). This matters because GitHub produces |
| 26 | +**no status at all** for a workflow skipped by a top-level `paths:` filter — a |
| 27 | +required context that never reports stays "Expected — Waiting" and blocks every |
| 28 | +non-native PR forever. See the `REQUIRED-SAFE DESIGN` header in |
| 29 | +`native-tests.yml`. |
| 30 | + |
| 31 | +`qa-gate` is a **commit status** the QATester posts at the end of its run |
| 32 | +(keyed to the head SHA). Because it is SHA-keyed, any new push lands on a SHA |
| 33 | +with no `qa-gate` status → the check returns to pending → QA must re-bless the |
| 34 | +new head. This enforces the spec-075 rule ("PASS valid only while PR head == |
| 35 | +qa_head_sha") in the merge button itself. |
| 36 | + |
| 37 | +## Activation (run after the workflow + scripts are on `main`) |
| 38 | + |
| 39 | +> Order matters: a required check that has **never produced a status** shows as |
| 40 | +> pending on every open PR and blocks normal merges immediately. Land |
| 41 | +> `native-tests.yml` and the scripts on `main` first, then **verify on a |
| 42 | +> non-native PR** (e.g. a dependency bump that touches none of the filtered |
| 43 | +> paths) that `swift-test` and `settings-parity` both report green (skipped) and |
| 44 | +> do **not** block it. Only after that confirmation, add the contexts to branch |
| 45 | +> protection. |
| 46 | +
|
| 47 | +Current required checks (do not drop them — the API call **replaces** the set): |
| 48 | + |
| 49 | +``` |
| 50 | +Lint, Unit Tests (ubuntu-latest, 1.25), Build (ubuntu-latest), |
| 51 | +Build (macos-latest), Build (windows-latest), Build Frontend, |
| 52 | +Validate PR title, Verify OpenAPI Artifacts |
| 53 | +``` |
| 54 | + |
| 55 | +Add the three new contexts (keeps `enforce_admins: false`): |
| 56 | + |
| 57 | +```bash |
| 58 | +gh api -X PATCH repos/:owner/:repo/branches/main/protection/required_status_checks \ |
| 59 | + -f strict=false \ |
| 60 | + -f 'contexts[]=Lint' \ |
| 61 | + -f 'contexts[]=Unit Tests (ubuntu-latest, 1.25)' \ |
| 62 | + -f 'contexts[]=Build (ubuntu-latest)' \ |
| 63 | + -f 'contexts[]=Build (macos-latest)' \ |
| 64 | + -f 'contexts[]=Build (windows-latest)' \ |
| 65 | + -f 'contexts[]=Build Frontend' \ |
| 66 | + -f 'contexts[]=Validate PR title' \ |
| 67 | + -f 'contexts[]=Verify OpenAPI Artifacts' \ |
| 68 | + -f 'contexts[]=swift-test' \ |
| 69 | + -f 'contexts[]=settings-parity' \ |
| 70 | + -f 'contexts[]=qa-gate' |
| 71 | +``` |
| 72 | + |
| 73 | +Stage `qa-gate` last — only after the QATester is posting it — so open PRs are |
| 74 | +not blocked on a status that nobody emits yet. Until then, add just `swift-test` |
| 75 | +and `settings-parity` — they report on every PR (green/skipped on non-native |
| 76 | +PRs) thanks to the required-safe design above, so they will not strand open PRs. |
| 77 | + |
| 78 | +## QATester contract |
| 79 | + |
| 80 | +The `mcpproxy-qa` skill ("Merge Gate" + "Native macOS Tray Testing" sections) |
| 81 | +requires QATester to: |
| 82 | + |
| 83 | +1. Treat every surface implied by the diff as mandatory — `native/macos/**` |
| 84 | + means the native tray (swift test green + behavioral assertions), never |
| 85 | + waived as "mirrors the frontend". |
| 86 | +2. Treat an interactive-surface `cannot_verify` as a **BLOCK**, not a low-risk |
| 87 | + pass (the MCP-1214 root cause). |
| 88 | +3. Post the gate status for the exact head SHA at the end of the run: |
| 89 | + `scripts/post-qa-gate-status.sh "$QA_HEAD_SHA" success|failure "..."`. |
| 90 | + |
| 91 | +## Known follow-up |
| 92 | + |
| 93 | +`native-tests.yml` skips a handful of pre-existing/environmental Swift test |
| 94 | +failures (AutoStart UserDefaults first-run, SSE-parser edge cases, a |
| 95 | +JSONEncoder behavior canary) so the gate is green today. Green those and remove |
| 96 | +the `--skip` flags to widen coverage. |
0 commit comments