Skip to content

Commit b2b9985

Browse files
claudeclaude
committed
test(release): boot a board built from the packages just published
0.21.0 was cut to fix a board that rendered raw message keys and had no CSS, and shipped with only the first of the two fixes. A stacked pull request merged into a base that had already merged, so its commits never reached main. Nothing noticed: main was green, the tag was coherent, the packages published, and the notes described a fix that was not in them. It was caught by grepping main by hand before re-pinning the Vercel template. The self-host template drifting three versions behind, serving the same message-keys bug to anyone clicking "Use this template", was the same shape. Every gate in this repository examines the repository. None of them look at what a user downloads. So the release pipeline now scaffolds from the create-meith it just published, installs from the real registry rather than from anything in this tree, and boots the result at both materialization depths — the same three assertions the workspace smoke makes. It runs after the npm job for that reason and `publish` waits on it, so a broken artefact stops the release being announced. It cannot un-publish a bad version and is not meant to; it makes the failure known in minutes rather than after someone deploys it. The boot and its assertions move to scripts/board-boot-check.mts so both smokes share one definition of a board working, rather than the published one drifting into a weaker check than the one it is meant to mirror.
1 parent 7766057 commit b2b9985

6 files changed

Lines changed: 248 additions & 84 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,50 @@ jobs:
169169
- run: pnpm --filter create-meith build
170170
- run: node scripts/npm-publish.mjs
171171

172+
# MEI-137: 0.21.0 was cut to fix a board that rendered raw message keys and
173+
# had no CSS, and shipped with only the first of the two fixes — a stacked
174+
# pull request merged into a base that had already merged, so its commits
175+
# never reached main. Nothing noticed: main was green, the tag was cut, the
176+
# packages published, and the release notes described a fix that was not in
177+
# them. Every other gate in this repository looks at the repository. This one
178+
# looks at what a user downloads: it scaffolds from the published
179+
# create-meith, installs from the real registry, and boots the result at both
180+
# materialization depths. It runs after npm for that reason and gates
181+
# `publish`, so a broken artefact stops the release being announced even
182+
# though it cannot stop it being published.
183+
published-board:
184+
name: A board built from the published packages boots
185+
needs: npm
186+
runs-on: ubuntu-latest
187+
services:
188+
postgres:
189+
image: postgres:18-alpine
190+
env:
191+
POSTGRES_USER: postgres
192+
POSTGRES_PASSWORD: postgres
193+
POSTGRES_DB: published_board_smoke
194+
ports: ['5432:5432']
195+
options: >-
196+
--health-cmd pg_isready --health-interval 10s
197+
--health-timeout 5s --health-retries 5
198+
steps:
199+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
200+
201+
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
202+
203+
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
204+
with:
205+
node-version: 26
206+
cache: pnpm
207+
208+
- run: pnpm install --frozen-lockfile
209+
210+
- name: Scaffold from the registry, build and boot at both depths
211+
env:
212+
DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/published_board_smoke
213+
run: |
214+
MEITH_VERSION="${GITHUB_REF_NAME#v}" pnpm published:board:smoke
215+
172216
# MEI-77: the framework base image a scaffolded board's own Dockerfile
173217
# starts FROM (docker/Dockerfile.base) — deps + framework layers only, no
174218
# board, no secrets. Needs [build, npm] rather than just `versions`: the
@@ -229,7 +273,7 @@ jobs:
229273
230274
publish:
231275
name: Tags, the release branch, and the draft notes
232-
needs: [build, npm, base-image]
276+
needs: [build, npm, base-image, published-board]
233277
runs-on: ubuntu-24.04
234278
steps:
235279
# Full history, not the default shallow clone: pushing over an existing

‎docs/release.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,10 +179,28 @@ broken promise, which is why the workflow drafts rather than publishes.
179179
does if one of those three was skipped as new-to-the-registry; re-run
180180
the workflow once it is published by hand, same as everywhere else in
181181
this pipeline;
182+
- **a board is scaffolded from the packages that were just published**,
183+
installed from the real registry rather than from anything in this
184+
tree, and booted at both materialization depths — the same three
185+
assertions the workspace smoke makes, that it renders `<main>`, that
186+
no message key reaches the page as text, and that the stylesheet
187+
carries rules for classes only the installed packages produce;
182188
- the `release` branch is fast-forwarded to the tag — refused if the
183189
tag is not descended from it, which is the guard against tagging a
184190
side branch;
185191
- the GitHub Release is drafted.
192+
193+
**Why a check that runs after publishing is worth having.** 0.21.0 was
194+
cut to fix a board that rendered raw message keys and shipped with only
195+
one of the two fixes in it: a stacked pull request had merged into a base
196+
that had already merged, so its commits never reached `main`. Nothing
197+
noticed — `main` was green, the tag was coherent, the packages published,
198+
and the notes described a fix that was not there. Every other gate in
199+
this repository examines the repository; this one examines what a user
200+
downloads. It cannot un-publish a bad version, and it is not meant to:
201+
`publish` waits on it, so a broken artefact stops the release being
202+
announced and tells you within minutes rather than after somebody
203+
deploys it.
186204
4. **Finish the draft.** Fill in the migration line, trim the generated
187205
notes to what an operator needs, publish.
188206

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@
5757
"vercel-template:gen": "tsx scripts/vercel-template-gen.mts",
5858
"vercel-template:gen:check": "tsx scripts/vercel-template-gen.mts --check",
5959
"board:workspace:smoke": "tsx scripts/board-workspace-smoke.mts",
60+
"published:board:smoke": "tsx scripts/published-board-smoke.mts",
6061
"board:deploy-kit:smoke": "tsx scripts/board-deploy-kit-smoke.mts",
6162
"board:eject:smoke": "tsx scripts/board-eject-smoke.mts",
6263
"perf": "tsx packages/testkit/src/load/run.ts",

‎scripts/board-boot-check.mts‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
import { spawn } from 'node:child_process'
2+
import { join } from 'node:path'
3+
4+
import defaultEnMessages from '../themes/default/src/messages/en.json' with { type: 'json' }
5+
import {
6+
assertBoardAssetsServe,
7+
assertMessagesResolve,
8+
assertStylesResolve,
9+
} from './board-smoke-assets.mts'
10+
11+
export const AT_ROOT_FLAG = '--at-root'
12+
export const AUTH_SECRET = 'smoke-test-auth-secret-32-bytes-min'
13+
export const TICK_SECRET = 'smoke-test-tick-secret-32-bytes-min'
14+
15+
async function waitForResponse(url: string, attempts: number): Promise<Response> {
16+
let lastError: unknown
17+
for (let attempt = 0; attempt < attempts; attempt += 1) {
18+
try {
19+
return await fetch(url)
20+
} catch (error) {
21+
lastError = error
22+
await new Promise((resolve) => setTimeout(resolve, 1000))
23+
}
24+
}
25+
throw new Error(`board-workspace-smoke: ${url} never answered: ${String(lastError)}`)
26+
}
27+
28+
export async function bootAndCheck(
29+
boardDir: string,
30+
port: string,
31+
atRoot: boolean,
32+
databaseUrl: string,
33+
) {
34+
const label = atRoot ? 'at the project root' : 'at .meith/app'
35+
const flag = atRoot ? [AT_ROOT_FLAG] : []
36+
37+
console.log(`== forum-web start ${label} ==`)
38+
const server = spawn(join(boardDir, 'node_modules/.bin/forum-web'), ['start', ...flag], {
39+
cwd: boardDir,
40+
detached: true,
41+
stdio: ['ignore', 'pipe', 'pipe'],
42+
env: {
43+
...process.env,
44+
PORT: port,
45+
DATABASE_URL: databaseUrl,
46+
DATA_SOURCE: 'postgres',
47+
AUTH_SECRET,
48+
TICK_SECRET,
49+
APP_URL: `http://127.0.0.1:${port}`,
50+
},
51+
})
52+
server.stdout?.on('data', (chunk) => process.stdout.write(chunk))
53+
server.stderr?.on('data', (chunk) => process.stderr.write(chunk))
54+
55+
function stopServer() {
56+
if (server.pid === undefined) return
57+
try {
58+
process.kill(-server.pid, 'SIGTERM')
59+
} catch {}
60+
setTimeout(() => {
61+
if (server.pid === undefined) return
62+
try {
63+
process.kill(-server.pid, 'SIGKILL')
64+
} catch {}
65+
}, 5000).unref()
66+
}
67+
68+
try {
69+
console.log('== waiting for it to answer / ==')
70+
const response = await waitForResponse(`http://127.0.0.1:${port}/`, 40)
71+
if (!response.ok) {
72+
throw new Error(`board-workspace-smoke: / answered ${response.status} (${label})`)
73+
}
74+
const body = await response.text()
75+
if (!body.includes('<main')) {
76+
throw new Error(`board-workspace-smoke: / answered but did not render <main> (${label})`)
77+
}
78+
assertMessagesResolve(body, Object.keys(defaultEnMessages))
79+
console.log(`== the board materialized ${label} rendered / ==`)
80+
81+
console.log('== confirming static assets and /sw.js actually serve ==')
82+
await assertBoardAssetsServe(`http://127.0.0.1:${port}`, body)
83+
console.log('== static assets and /sw.js served correctly ==')
84+
85+
console.log('== confirming the stylesheet actually styles what rendered ==')
86+
await assertStylesResolve(`http://127.0.0.1:${port}`, body)
87+
console.log('== every class the board rendered has a rule ==')
88+
} finally {
89+
stopServer()
90+
}
91+
}

‎scripts/board-workspace-smoke.mts‎

Lines changed: 3 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,7 @@ import { tmpdir } from 'node:os'
4242
import { join } from 'node:path'
4343

4444
import defaultEnMessages from '../themes/default/src/messages/en.json' with { type: 'json' }
45-
import {
46-
assertBoardAssetsServe,
47-
assertMessagesResolve,
48-
assertStylesResolve,
49-
} from './board-smoke-assets.mts'
45+
import { AT_ROOT_FLAG, AUTH_SECRET, bootAndCheck, TICK_SECRET } from './board-boot-check.mts'
5046
import { packClosure } from './pack-workspace-closure.mts'
5147
import { ROOT } from './workspace-packages.mjs'
5248

@@ -58,9 +54,6 @@ if (!DATABASE_URL) {
5854

5955
const PORT = process.env.SMOKE_PORT ?? '3999'
6056
const AT_ROOT_PORT = process.env.SMOKE_AT_ROOT_PORT ?? String(Number(PORT) + 1)
61-
const AT_ROOT_FLAG = '--at-root'
62-
const AUTH_SECRET = 'smoke-test-auth-secret-32-bytes-min'
63-
const TICK_SECRET = 'smoke-test-tick-secret-32-bytes-min'
6457

6558
function run(
6659
command: string,
@@ -133,79 +126,6 @@ async function pointAtTarballs(boardDir: string, tarballs: ReadonlyMap<string, s
133126
await writeFile(packageJsonPath, `${JSON.stringify(manifest, null, 2)}\n`)
134127
}
135128

136-
async function waitForResponse(url: string, attempts: number): Promise<Response> {
137-
let lastError: unknown
138-
for (let attempt = 0; attempt < attempts; attempt += 1) {
139-
try {
140-
return await fetch(url)
141-
} catch (error) {
142-
lastError = error
143-
await new Promise((resolve) => setTimeout(resolve, 1000))
144-
}
145-
}
146-
throw new Error(`board-workspace-smoke: ${url} never answered: ${String(lastError)}`)
147-
}
148-
149-
async function bootAndCheck(boardDir: string, port: string, atRoot: boolean) {
150-
const label = atRoot ? 'at the project root' : 'at .meith/app'
151-
const flag = atRoot ? [AT_ROOT_FLAG] : []
152-
153-
console.log(`== forum-web start ${label} ==`)
154-
const server = spawn(join(boardDir, 'node_modules/.bin/forum-web'), ['start', ...flag], {
155-
cwd: boardDir,
156-
detached: true,
157-
stdio: ['ignore', 'pipe', 'pipe'],
158-
env: {
159-
...process.env,
160-
PORT: port,
161-
DATABASE_URL,
162-
DATA_SOURCE: 'postgres',
163-
AUTH_SECRET,
164-
TICK_SECRET,
165-
APP_URL: `http://127.0.0.1:${port}`,
166-
},
167-
})
168-
server.stdout?.on('data', (chunk) => process.stdout.write(chunk))
169-
server.stderr?.on('data', (chunk) => process.stderr.write(chunk))
170-
171-
function stopServer() {
172-
if (server.pid === undefined) return
173-
try {
174-
process.kill(-server.pid, 'SIGTERM')
175-
} catch {}
176-
setTimeout(() => {
177-
if (server.pid === undefined) return
178-
try {
179-
process.kill(-server.pid, 'SIGKILL')
180-
} catch {}
181-
}, 5000).unref()
182-
}
183-
184-
try {
185-
console.log('== waiting for it to answer / ==')
186-
const response = await waitForResponse(`http://127.0.0.1:${port}/`, 40)
187-
if (!response.ok) {
188-
throw new Error(`board-workspace-smoke: / answered ${response.status} (${label})`)
189-
}
190-
const body = await response.text()
191-
if (!body.includes('<main')) {
192-
throw new Error(`board-workspace-smoke: / answered but did not render <main> (${label})`)
193-
}
194-
assertMessagesResolve(body, Object.keys(defaultEnMessages))
195-
console.log(`== the board materialized ${label} rendered / ==`)
196-
197-
console.log('== confirming static assets and /sw.js actually serve ==')
198-
await assertBoardAssetsServe(`http://127.0.0.1:${port}`, body)
199-
console.log('== static assets and /sw.js served correctly ==')
200-
201-
console.log('== confirming the stylesheet actually styles what rendered ==')
202-
await assertStylesResolve(`http://127.0.0.1:${port}`, body)
203-
console.log('== every class the board rendered has a rule ==')
204-
} finally {
205-
stopServer()
206-
}
207-
}
208-
209129
async function main() {
210130
const tarballDir = await mkdtemp(join(tmpdir(), 'board-workspace-smoke-tarballs-'))
211131
const scaffoldParent = await mkdtemp(join(tmpdir(), 'board-workspace-smoke-board-'))
@@ -264,7 +184,7 @@ async function main() {
264184
)
265185
}
266186

267-
await bootAndCheck(boardDir, PORT, false)
187+
await bootAndCheck(boardDir, PORT, false, DATABASE_URL)
268188

269189
console.log('== a second board, materialized the way Vercel deploys it ==')
270190
const atRootDir = await scaffoldBoard(scaffoldParent, 'smoke-board-at-root')
@@ -274,7 +194,7 @@ async function main() {
274194
cwd: atRootDir,
275195
env: { ...process.env, DATABASE_URL: '', DATA_SOURCE: '' },
276196
})
277-
await bootAndCheck(atRootDir, AT_ROOT_PORT, true)
197+
await bootAndCheck(atRootDir, AT_ROOT_PORT, true, DATABASE_URL)
278198
} finally {
279199
await rm(tarballDir, { recursive: true, force: true })
280200
await rm(scaffoldParent, { recursive: true, force: true })

0 commit comments

Comments
 (0)