Skip to content

Commit d9f159f

Browse files
committed
docs(skills): fix CI/CD pipeline docs — source-verified from actual workflow files
All trigger tables, branch/tag maps, and pipeline diagrams were wrong or incomplete. Fixed against the actual .github/workflows/ source: - workflow-map.md: testing branch is the PRIMARY :testing publish path, not just a source branch for PRs. Added correct branch→tag mapping table verified from publish.yml setup job logic. Fixed build.yml trigger to include push: main/next/testing (was missing entirely). - ci.md skill: replaced broken Common Rationalizations table (had workflow names as "rationalizations") with a correct Workflow Quick Reference table. Added push column to Trigger Behavior table. Fixed duplicate step numbers in Core Process. - release-promotion.md: fixed Promotion Map (was showing main merge as the start; testing push is the actual start). Fixed duplicate step 2 in Core Process. - quickstart.md: fixed duplicate rule 2 in Always Rules. - docs/ci.md: fixed build job trigger row (said 'schedule' which does not exist; was missing push trigger). Fixed publish pipeline diagram to show main|testing|next, not just main. Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
1 parent e513d44 commit d9f159f

5 files changed

Lines changed: 104 additions & 52 deletions

File tree

docs/ci.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,17 @@
66
|---|---|---|
77
| `validate` | `pull_request` | `bst show` — graph + patch check (~15 min) |
88
| `e2e` | `pull_request` when `elements/`, `files/`, `patches/`, `Justfile`, or `project.conf` changed | Smoke test in QEMU via projectbluefin/testsuite |
9-
| `build` | `merge_group`, `schedule`, `workflow_dispatch` (skips on `pull_request`) | Full OCI build (~60–90 min) |
9+
| `build` | `push: main/next/testing` (paths-ignore: `.github/workflows/**`, `docs/**`, `**.md`, `AGENTS.md`), `merge_group`, `workflow_dispatch` skips on `pull_request` | Full OCI build (~60–90 min) |
1010
| `build-aarch64` | disabled | ARM64 — pending investigation |
1111

1212
## Publish pipeline (publish.yml)
1313

1414
`build` success on main/testing/next triggers publish.yml via `workflow_run`:
1515

1616
```
17-
build.yml (main) → [workflow_run] → publish.yml
18-
setup → publish-image (matrix) → promote
19-
└──────────────→ publish-sbom
17+
build.yml (main|testing|next) → [workflow_run] → publish.yml
18+
setup → publish-image (matrix) → promote (:testing or :next)
19+
└──────────────→ publish-sbom
2020
```
2121

2222
| Job | What |

docs/skills/ci.md

Lines changed: 58 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,45 @@ Use this skill when the task mentions:
3030
- OCI image contents or layer assembly → `oci-layers.md`
3131
- Normal PR review → `pr-review.md`
3232

33+
## ⚠️ Builder Discipline — Read Before Doing Anything
34+
35+
**ONE BST build at a time. Always.**
36+
37+
Before merging, pushing, or dispatching any workflow, run the mandatory pre-flight:
38+
39+
```bash
40+
gh run list --repo projectbluefin/dakota --limit 30 \
41+
--json databaseId,status,name,headBranch \
42+
| python3 -c "
43+
import json, sys
44+
runs = json.load(sys.stdin)
45+
active = [r for r in runs if r['status'] in ('in_progress', 'queued', 'pending', 'waiting')]
46+
if active:
47+
print(f'BLOCKED: {len(active)} active run(s). Cancel all before proceeding:')
48+
for r in active:
49+
print(f' gh run cancel {r[\"databaseId\"]} --repo projectbluefin/dakota # {r[\"name\"]} [{r[\"headBranch\"]}]')
50+
else:
51+
print('OK: field is clear')
52+
"
53+
```
54+
55+
If output is not `OK: field is clear`**cancel every listed run first**.
56+
57+
**Cache-warm is not exempt.** It shares the same `ubuntu-24.04` runner pool and CAS write bandwidth as real builds. Two concurrent BST jobs do not halve wall time — they more than double it and risk 6-hour timeouts with zero elements cached.
58+
59+
The rationalizations that have caused real production failures:
60+
- "Cache-warm is additive, it helps the build" → **No. Cancel it.**
61+
- "This is almost done, just a few minutes" → **Cancel it. You don't know that.**
62+
- "It's a different branch, it won't interfere" → **Same runners and CAS. Cancel it.**
63+
- "I cancelled one, that's enough" → **Cancel ALL. Re-run pre-flight.**
64+
3365
## Core Process
3466

35-
1. **Classify the failure before reading logs.**
67+
1. **Run the mandatory pre-flight above. Verify `OK: field is clear`.**
68+
2. **Classify the failure before reading logs.**
3669
- *Which workflow?* `build`, `publish`, `promote`, `release`, `e2e`, `merge queue`
3770
- *Which phase?* trigger, setup, reusable workflow call, build/export, boot, smoke, promotion
38-
2. **Load one next skill, not five.**
71+
3. **Load one next skill, not five.**
3972
- Need workflow/trigger map → `workflow-map.md`
4073
- Need reusable workflow / permissions / cache-dir weirdness → `ci-tooling.md`
4174
- Need boot-check / smoke / testsuite behavior → `e2e-ci.md`
@@ -57,26 +90,33 @@ Use this skill when the task mentions:
5790
| conflicting chore PRs, stale queue branches | `merge-queue.md` |
5891
| historical edge cases and deep cuts | `ci-reference.md` |
5992

60-
## Common Rationalizations
93+
## Workflow Quick Reference
6194

62-
| Rationalization | Reality |
63-
|---|---|
64-
| `.github/workflows/build.yml` | BST build + push artifacts to remote CAS. Fires on `merge_group` and `workflow_dispatch` only (no schedule). Does NOT push to GHCR directly. |
65-
| `.github/workflows/publish.yml` | 3-stage pipeline: setup → publish → promote. Pulls artifact from CAS, exports OCI, pushes `:$sha`, signs, attests, then immediately promotes to `:testing` on every successful merge. No e2e gate — that lives only in the weekly promotion. |
66-
| `.github/workflows/promote-testing-to-main.yml` | Thin caller for `reusable-promote.yml` in `projectbluefin/actions`. Fires on `push: testing`, nightly schedule (23:00 UTC), and `workflow_dispatch`. Opens or updates the promotion PR that gates `:testing``:stable`. |
67-
| `.github/workflows/execute-release.yml` | Fires on `push: main` + `workflow_dispatch`. A `check-trigger` job reads the squash-merge commit message — only proceeds when it starts with `ci: promote testing images to stable`. Calls `reusable-execute-release.yml` (copies image tags) then `reusable-release.yml` (generates GitHub Release + SBOM diff). |
68-
| `.github/workflows/e2e.yml` | Smoke test via projectbluefin/testsuite. Fires on PR; `should-run` job skips the test when no image-affecting paths changed. |
69-
| `.github/workflows/vulnerability-scan.yml` | Weekly Monday 08:00 UTC CVE scan via `reusable-vulnerability-scan.yml`. Also available as `workflow_dispatch` with optional `image_ref` input. Results surface in the GitHub Security tab. |
95+
| Workflow | Trigger | What it does |
96+
|---|---|---|
97+
| `build.yml` | `push: main/next/testing` (paths-ignore: docs/workflows/md), `merge_group`, `workflow_dispatch` — NOT `pull_request` | BST build artifacts into remote CAS. Does NOT push to GHCR. `validate` job runs on `pull_request` only; `build` job runs on everything else. |
98+
| `publish.yml` | `workflow_run` from `build.yml` (branches: main, next, testing, + their gh-readonly-queue/* paths) | Export from CAS → push `:$sha` → sign/attest → promote to `:testing`/`:next`. No build happens here. |
99+
| `promote-testing-to-main.yml` | `push: testing`, `schedule: Tue 04:00 UTC`, `workflow_dispatch` | Opens/updates promotion PR from testing into main. |
100+
| `pr-release-gate.yml` | `pull_request` to `main` | Gates the promotion PR via cosign verify of `:testing`. |
101+
| `execute-release.yml` | `push: main`, `workflow_dispatch` | Reads commit message — proceeds only when it starts with `ci: promote testing images to stable`. Copies tags to `:latest`/`:stable`, creates GitHub Release. |
102+
| `cache-warm.yml` | `schedule: Mon/Thu 06:00 UTC`, `workflow_dispatch` | Pre-warms remote CAS. Two parallel jobs (x86_64, aarch64), `continue-on-error: true`. **Not exempt from pre-flight — cancel before any real build.** |
70103

71104
## Trigger Behavior
72105

73-
| Behavior | pull_request | merge_group | workflow_dispatch | schedule |
74-
|---|---|---|---|---|
75-
| `validate` job | Yes | No | No | No |
76-
| `e2e` job | Yes (change-detected) | No | Yes | No |
77-
| `build` job | No | Yes | Yes | No |
78-
| `cache-warm` job | No | No | Yes | Yes (Mon/Thu 06:00 UTC) |
79-
| Push to GHCR? | No | Via publish.yml | Via publish.yml | No |
106+
| Job | pull_request | push main/next/testing | merge_group | workflow_dispatch | schedule |
107+
|---|---|---|---|---|---|
108+
| `validate` | Yes | No | No | No | No |
109+
| `e2e` | Yes (change-detected) | No | No | Yes | No |
110+
| `build` | No | Yes (paths-ignore) | Yes | Yes | No |
111+
| `cache-warm` | No | No | No | Yes | Mon/Thu 06:00 UTC |
112+
| Push to GHCR? | No | Via publish.yml | Via publish.yml | Via publish.yml | No |
113+
114+
**push paths-ignore:** `.github/workflows/**`, `docs/**`, `**.md`, `AGENTS.md` — doc/workflow-only pushes do NOT trigger a build. This is intentional; it means a CI-only commit advancing the branch HEAD will leave no build artifact for that SHA.
115+
116+
**Branch → tag mapping** (verified from publish.yml source):
117+
- `main` or `gh-readonly-queue/main/*``:testing`
118+
- `testing` or `gh-readonly-queue/testing/*``:testing`
119+
- `next` or `gh-readonly-queue/next/*``:next`
80120

81121
**PR path:** `validate` + `e2e` (change-detected) — zero remote execution. ~15 min cached, ~30 min cold.
82122

docs/skills/quickstart.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,13 @@ Use when:
3838

3939
## Always Rules
4040

41-
1. **Run the CI pre-flight before any merge or push.** (See Hard Rule #9 in `.github/copilot-instructions.md`.)
41+
1. **Run the CI pre-flight before any merge, push, or workflow dispatch.** (See Hard Rule #9 in `.github/copilot-instructions.md`.)
4242
2. Run `just --list` first.
43-
2. Use `just bst ...`, not bare `bst`.
44-
3. Grep all references before removing a package or file.
45-
4. Add new package elements to the correct stack.
46-
5. Validate before opening the PR.
47-
6. Push to `upstream`, never the fork workflow by accident.
43+
3. Use `just bst ...`, not bare `bst`.
44+
4. Grep all references before removing a package or file.
45+
5. Add new package elements to the correct stack.
46+
6. Validate before opening the PR.
47+
7. Push to `upstream`, never the fork.
4848

4949
## Never Rules
5050

docs/skills/release-promotion.md

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -42,32 +42,31 @@ Use when the task mentions:
4242
- open/update promotion PR
4343
- gate the promotion PR
4444
- execute stable release after merge
45-
2. **Check whether the gate is real or infrastructure.**
45+
3. **Check whether the gate is real or infrastructure.**
4646
- `action_required` on a promotion PR often means the gate is waiting on policy or verification, not that the YAML crashed.
47-
3. **For promotion PR workflows, inspect reusable caller permissions first.**
48-
4. **Do not add e2e back into the promotion PR path.**
47+
4. **For promotion PR workflows, inspect reusable caller permissions first.**
48+
5. **Do not add e2e back into the promotion PR path.**
4949
Dakota intentionally gates stable at the later human-approved release stage.
50-
5. **Automatic promotion cadence is Tuesday 04:00 UTC.**
50+
6. **Automatic promotion cadence is Tuesday 04:00 UTC.**
5151
That schedule re-evaluates the promotion PR for the weekly stable cut.
52-
6. **Manual `workflow_dispatch` must preserve queue behavior.**
52+
7. **Manual `workflow_dispatch` must preserve queue behavior.**
5353
`use_merge_queue` should be true for `schedule` and `workflow_dispatch`, but
5454
not for ordinary `push` refreshes from `testing`.
55-
7. **For manual recovery, re-run the failed publish/promote workflow that owns the stage, not some nearby check.**
55+
8. **For manual recovery, re-run the failed publish/promote workflow that owns the stage, not some nearby check.**
5656

5757
## Promotion Map
5858

5959
```text
60-
main merge
61-
→ build.yml
62-
→ publish.yml
63-
→ :testing
64-
→ push testing / nightly / manual
60+
push to testing (BST-affecting paths)
61+
→ build.yml (build job)
62+
→ publish.yml (workflow_run)
63+
→ :testing tag published to GHCR
6564
→ promote-testing-to-main.yml
66-
→ auto/promote-testing-to-main PR
67-
→ pr-release-gate.yml
68-
merge promotion PR
69-
→ execute-release.yml
70-
→ :latest / :stable + release notes
65+
→ auto/promote-testing-to-main PR
66+
→ pr-release-gate.yml (cosign verify :testing)
67+
→ auto-merge → push to main
68+
→ execute-release.yml (commit message gate)
69+
→ :latest / :stable + GitHub Release
7170
```
7271

7372
## Hard Rules

docs/skills/workflow-map.md

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ merge promotion PR to main
7272

7373
| Workflow | Owns | Normal trigger |
7474
|---|---|---|
75-
| `.github/workflows/build.yml` | BST build into remote CAS | `merge_group`, `workflow_dispatch` |
75+
| `.github/workflows/build.yml` | BST build into remote CAS | `push: main/next/testing` (paths-ignore: docs, workflows, md), `merge_group`, `workflow_dispatch`. `validate` job runs on `pull_request` only; `build` job skips `pull_request`. |
7676
| `.github/workflows/publish.yml` | export, sign, boot-check, promote tags | `workflow_run` from build |
7777
| `.github/workflows/publish-smoke.yml` | observational smoke only | `workflow_run` from publish |
7878
| `.github/workflows/e2e.yml` | PR-facing testsuite check | `pull_request` |
@@ -83,13 +83,26 @@ merge promotion PR to main
8383

8484
## Branch / Tag Map
8585

86-
| Branch | Result |
87-
|---|---|
88-
| `main` | merged changes build, publish `:$sha`, then promote to `:testing` |
89-
| `testing` | source branch for promotion PRs into `main` |
90-
| `next` | rolling GNOME master stream; publish to `:next` and `:btw`, never stable. No PR requirement on branch protection (dev stream, direct push from `sync-next-from-main` is intentional) |
91-
| `gh-readonly-queue/main/*` | merge-queue build path for `main` |
92-
| `gh-readonly-queue/next/*` | merge-queue build path for `next` |
86+
| Branch | Trigger | Published tag(s) | Notes |
87+
|---|---|---|---|
88+
| `testing` | `push` (BST-affecting paths only) | `:testing` | **Primary `:testing` publish path.** Every BST-affecting push builds → publishes → promotes. Doc/workflow-only pushes are ignored (paths-ignore). |
89+
| `main` | merge of promotion PR | `:latest`, `:stable` | Only via `execute-release.yml` and only when commit message starts with `ci: promote testing images to stable`. Normal merges do NOT produce a new tag. |
90+
| `next` | `push` or `sync-next-from-main` dispatch | `:next`, `:btw` | Rolling GNOME master; never stable. No PR requirement on branch protection. |
91+
| `gh-readonly-queue/main/*` | merge-queue | (build only, no tag) | Gate before merge to `main`. |
92+
| `gh-readonly-queue/next/*` | merge-queue | (build only, no tag) | Gate before merge to `next`. |
93+
94+
**What testing does (not just PRs):**
95+
```
96+
push to testing (BST-affecting)
97+
→ build.yml (build job)
98+
→ publish.yml (workflow_run)
99+
→ :testing tag published to GHCR
100+
→ promote-testing-to-main.yml
101+
→ opens/updates auto/promote-testing-to-main PR
102+
→ pr-release-gate.yml gates it
103+
→ auto-merge → push to main
104+
→ execute-release.yml → :stable / :latest
105+
```
93106

94107
## Common Rationalizations
95108

0 commit comments

Comments
 (0)