-
-
Notifications
You must be signed in to change notification settings - Fork 277
402 lines (350 loc) · 17.2 KB
/
Copy pathci.yml
File metadata and controls
402 lines (350 loc) · 17.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
name: CI
on:
pull_request:
paths-ignore:
- 'docs/**'
- 'website/**'
- 'README.md'
- 'AGENTS.md'
- 'CHANGELOG.md'
- 'CONTEXT.md'
- 'CONTRIBUTING.md'
- 'LICENSE'
- 'SECURITY.md'
- '.github/actions/build-docs/action.yml'
- '.github/workflows/deploy.yml'
- '.github/workflows/pr-preview.yml'
- '.github/workflows/pr-preview-cleanup.yml'
push:
branches:
- main
permissions:
contents: read
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# The Swift trailing-comma assertion is text-only and runs before the toolchain setup, so a
# grep failure does not wait on an install. The DI-seams check below needs a real TypeScript
# runtime (#1976 / PR #2006), so it runs after Setup toolchain instead.
lint:
name: Lint & Format
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# #1976: ripgrep is never installed on ubuntu-latest, so `rg` failed with "command not
# found" (exit 127) on every run, and `if rg ...; then ... fi` cannot distinguish that
# from "no matches" (exit 1) — both read as false, so the step passed without the
# assertion ever executing. Rewritten against `grep`, which every runner ships, with the
# match/no-match/error exit codes handled explicitly so a broken scan fails loudly instead
# of silently passing.
- name: Disallow trailing commas before closing parenthesis in Swift
run: |
mapfile -d '' -t swift_files < <(git ls-files -z -- 'apple/runner' | grep -z '\.swift$')
if [ "${#swift_files[@]}" -eq 0 ]; then
echo "No apple/runner/*.swift files are tracked; the trailing-comma check has nothing to scan." >&2
exit 1
fi
set +e
grep -PzoH ',\s*\n\s*\)' "${swift_files[@]}"
status=$?
set -e
if [ "$status" -eq 0 ]; then
echo "Found trailing commas before ')' in Swift files. This syntax requires Swift 6.1+ and breaks older Xcode toolchains."
exit 1
elif [ "$status" -ne 1 ]; then
echo "grep exited $status while scanning apple/runner for trailing commas; treating an unreadable scan as a failure instead of a silent pass."
exit 1
fi
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
# Same false-green shape as the Swift check above (#1976). An earlier revision of this
# gate (PR #2006, first review pass) fixed the exit-code handling but kept the ban/allow
# decision as a regex that exempted matches by the *spelling* of the typeof target
# (`typeof fetch` always passed, SCREAMING_SNAKE_CASE targets always passed) — a name-based
# semantic allowlist that would silently pass a new, genuinely test-only `typeof fetch` seam
# anywhere in the tree while banning an equally legitimate seam under any other name.
# scripts/di-seams instead checks each match against an explicit, typed, per-site allowlist
# (scripts/di-seams/approved.ts) keyed by (file, field, typeof-target): a triple is exempt
# only because it was individually reviewed and named, never because of how it is spelled.
# The gate fails just as hard on a stale approval (one whose triple no longer matches
# anything) as on an unapproved seam, so the allowlist can't drift out of sync with the code
# it describes. See scripts/di-seams/model.ts and its tests.
- name: Fail if test-only DI seams reappear in production code
uses: ./.github/actions/run-gate
with: { gate: di-seams }
- name: Run oxlint
uses: ./.github/actions/run-gate
with: { gate: lint }
- name: Check formatting
uses: ./.github/actions/run-gate
with: { gate: format }
# Structural guards share one checkout and install. Every gate stays an
# independently named step with its own failure message; the gate manifest
# derives lane ownership from these declarations, so merging jobs changes
# nothing it asserts.
repo-guards:
name: Repo Guards
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
# The layering ratchets (R6/R9/R10) measure the merge-base with origin/main, which a
# shallow checkout cannot reach.
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
# The layering gate parses production sources with `oxc-parser`, so
# dependencies are required; keep install-deps enabled.
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Check import-direction DAG
# Structured import-direction lint over the resolved graph. See
# scripts/layering/check.ts and CONTEXT.md (Architecture: folder DAG +
# layering lint).
uses: ./.github/actions/run-gate
with: { gate: layering }
# Model tests for the dependency-graph report and its blast-radius query. The report
# reads the gate's model (scripts/layering/model.ts) and applies the gate's own R6
# counting rule, so it is not a second measurement of the R6 ratchet.
- name: Check the depgraph report model
uses: ./.github/actions/run-gate
with: { gate: depgraph }
# Tests for the TMPDIR redirection itself.
#
# Deliberately NOT in Coverage next to `check:tmpdir-leaks`, where the subject matter
# would put it: vitest-tmpdir-global-setup.test.ts proves the lifecycle by spawning a
# real nested `vitest run`, and the Coverage lane already loses runs to
# `[vitest-pool]: Worker forks emitted error` when a fork is slow to terminate.
# Starting a nested Vitest seconds before the full instrumented suite is a contention
# risk with nothing to gain.
- name: Check the tmpdir redirection model
uses: ./.github/actions/run-gate
with: { gate: tmpdir-leaks-model }
# The selector is fail-open and advisory (GitHub CI stays authoritative),
# so the gate only guards the derivation model.
- name: Check affected-selector model
uses: ./.github/actions/run-gate
with: { gate: affected-selector }
# The gate-of-gates (#1429). It shares this job because it validates the
# same artifact the selector is built on — CHECK_CATALOG's `ciJobs` — and
# because a gate that proves the other gates are wired must not be the one
# gate sitting in its own job, green on its own. Deterministic and
# network-free: every input is a file in the checkout.
- name: Check the gate manifest model
uses: ./.github/actions/run-gate
with: { gate: gate-manifest-model }
- name: Check every gate is owned, wired, and reachable
uses: ./.github/actions/run-gate
with: { gate: gate-manifest }
# Same family as the manifest above — a CI selection that has stopped selecting what
# it claims. ios.yml runs a hand-written subset of the runner XCTest methods through an
# `-only-testing:` list, and xcodebuild treats an identifier that matches nothing as
# an empty selection rather than an error, so a rename drops a test with no signal —
# in both directions, since a typo in xctest-nightly.yml's `-skip-testing:` entry
# re-arms a 24-hour hang. Parse-only, no Xcode (#1781 A7).
- name: Check the PR XCTest selection still names real tests
uses: ./.github/actions/run-gate
with: { gate: xctest-selection }
# Layers 1-2 of the conformance oracle: replay the JVM-generated fixtures
# against the live engine. Deterministic and Java-free — the generated
# fixtures are checked in and only regenerated on an upstream-pin bump. The
# device-backed layer 3 runs on the scheduled conformance-differential
# workflow. See scripts/maestro-conformance/README.md. Unlike the guards
# above, this parses corpus flows with the live engine and imports the
# `yaml` package — covered by the shared install above.
- name: Verify Maestro conformance fixtures
uses: ./.github/actions/run-gate
with: { gate: maestro-conformance }
# server.json/smithery.yaml drift otherwise surfaces at publish time (where
# publish-mcp-registry.yml duplicates the same command). Parse-only.
- name: Check MCP registry metadata is in sync
uses: ./.github/actions/run-gate
with: { gate: mcp-metadata }
# Slowest guard (~3 min), so it runs last and structural failures surface
# before it. Runs on plain Node via the shared install above.
- name: Check numeric ranges
uses: ./.github/actions/run-gate
with: { gate: freerange }
# History-backed compatibility gates share one deep checkout: fallow,
# replay-compat, and daemon-wire-compat all need either full history or tags,
# which no shallow unit lane can read.
compat-provenance:
name: Compatibility & Provenance
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
fetch-tags: true
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run Fallow audit
env:
FALLOW_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
uses: ./.github/actions/run-gate
with:
gate: fallow
args: |
--base
${{ env.FALLOW_BASE }}
- name: Check for production-unused exports
uses: ./.github/actions/run-gate
with: { gate: production-exports }
# The frozen replay-compat corpus (#1417) claims each entry was published by
# a released tag. Only a full-history checkout can re-derive that claim.
- name: Verify corpus entries against their released blobs
uses: ./.github/actions/run-gate
with: { gate: replay-compat }
- name: Verify the wire-compat rules
uses: ./.github/actions/run-gate
with: { gate: wire-compat-model }
# The daemon RPC wire ledger (#1432) is compared against the ledger as it
# stood at the last RELEASED tag, which only a tagged checkout can read.
# The shallow Repo Guards lane holds the ledger to its source; this step
# holds it to the last release.
- name: Compare the daemon RPC wire surface against the last released tag
uses: ./.github/actions/run-gate
with: { gate: daemon-wire-compat }
# Typecheck and package verification share one checkout and install; both
# need the default toolchain only, and the package step re-pins Node itself.
typecheck-package:
name: Typecheck & Package
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run typecheck
uses: ./.github/actions/run-gate
with: { gate: typecheck }
- name: Build CLI
uses: ./.github/actions/run-gate
with: { gate: build }
- name: Verify emitted chunk ownership
uses: ./.github/actions/run-gate
with: { gate: bundle-owner-files }
# The build runs on the default toolchain Node and the package is verified on the minimum
# supported Node, so this job covers what a user on `engines.node` floor actually installs.
- name: Setup Node.js 22.12
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version: '22.12'
# Packs, lints the tarball with publint/attw, installs it outside the workspace, and imports
# every published entry point before running the CLI. See scripts/check-package.ts.
#
# Runs the script directly rather than through `pnpm check:package`: the repo's pinned pnpm
# requires Node >= 22.13 and refuses to start on the 22.12 floor this job exists to cover. The
# gate itself only needs `node` and `npm`, so it is the package.json script minus the launcher.
- name: Verify the published package on Node.js 22.12
run: node --experimental-strip-types scripts/check-package.ts
# Runs the full unit + provider-integration suites under coverage with
# thresholds, so a separate unit-tests job would rerun the same tests.
coverage:
name: Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Test changed-line coverage gate
uses: ./.github/actions/run-gate
with: { gate: coverage-model }
- name: Run coverage
id: run-coverage
env:
OUTPUT_ECONOMY_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
uses: ./.github/actions/run-gate
with: { gate: unit-ci }
# The TMPDIR redirection both test lanes depend on (#1593/#1595). The check is a real
# package script that no workflow ran: it is reachable only through `check:unit`, an
# aggregate CI never invokes, so a leak regression could not fail a PR. Placed here
# because this is the lane whose instrumented suite would leak a run directory.
- name: Check for leaked temp directories
uses: ./.github/actions/run-gate
with: { gate: tmpdir-leaks }
# Reuses the lcov the coverage step just wrote (never runs coverage twice)
# and fails when changed-line coverage < the threshold in
# scripts/coverage-changed/model.ts. The `coverage-waiver` PR label maps to
# the waiver env, which skips the failure but still prints the numbers.
# Gated on the coverage step's own outcome (#1781 A5): when `Run coverage`
# fails, lcov.info is never written, so this step would just re-report that
# failure as its own red ("no lcov report") instead of a coverage verdict.
- name: Enforce changed-line coverage gate
if: steps.run-coverage.outcome == 'success' && github.event_name == 'pull_request'
env:
AGENT_DEVICE_COVERAGE_WAIVER: ${{ contains(github.event.pull_request.labels.*.name, 'coverage-waiver') }}
uses: ./.github/actions/run-gate
with:
gate: coverage
args: |
--base
${{ github.event.pull_request.base.sha }}
integration:
name: Integration Tests
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run integration tests
run: pnpm clean:daemon
- name: Execute integration tests
uses: ./.github/actions/run-gate
with: { gate: integration-node }
- name: Run seeded concurrency torture lane (fast PR sweep)
# #1416's nightly torture lane lives under test/integration/nightly/, out
# of the test:integration:node glob, so this is a *deliberate* fast PR
# sweep (TORTURE_RUNS default 128 seeds, ~sub-second) — not an accidental
# glob inclusion. The Concurrency Torture Nightly workflow sweeps a much
# larger seed range on schedule.
uses: ./.github/actions/run-gate
with: { gate: concurrency-torture }
- name: Run provider-backed integration tests
uses: ./.github/actions/run-gate
with: { gate: provider-integration }
- name: Check Provider-backed integration architecture progress
uses: ./.github/actions/run-gate
with: { gate: integration-progress }
# A build-cache lookup outage must degrade setup-fixture-app to an inline
# build, not fail the caller. This drives that step's real shell against a
# failing `gh`.
- name: Setup-fixture-app cache-failure fallback
uses: ./.github/actions/run-gate
with: { gate: fixture-fallback }
# The trusted-artifact contract the device lanes rely on to decide a cached fixture
# app is the one this commit expects. A real test file that no workflow ran.
- name: Check the trusted fixture-artifact contract
uses: ./.github/actions/run-gate
with: { gate: fixture-cache }
# Shares this job's ubuntu toolchain. AGENT_DEVICE_WEB_E2E is step-scoped
# so it cannot leak into the node/provider suites above.
- name: Run live web smoke
env:
AGENT_DEVICE_WEB_E2E: '1'
run: pnpm clean:daemon
- name: Execute live web smoke
env:
AGENT_DEVICE_WEB_E2E: '1'
uses: ./.github/actions/run-gate
with: { gate: web-smoke }
- name: Upload web smoke artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: web-smoke-artifacts
if-no-files-found: ignore
path: |
test/artifacts/web/**