Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/track-bst-sources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ jobs:
private-key: ${{ secrets.MERGERAPTOR_PRIVATE_KEY }}

- name: Setup Just
uses: taiki-e/install-action@9bcaee1dcae34154180f412e2fa69355a7cda9f6 # v2
uses: taiki-e/install-action@16b05812d776ae1dfaabc8277e421fb6d2506419 # v2
with:
tool: just

Expand Down Expand Up @@ -328,7 +328,7 @@ jobs:
token: ${{ steps.app-token.outputs.token }}

- name: Setup Just
uses: taiki-e/install-action@9bcaee1dcae34154180f412e2fa69355a7cda9f6 # v2
uses: taiki-e/install-action@16b05812d776ae1dfaabc8277e421fb6d2506419 # v2
with:
tool: just

Expand Down Expand Up @@ -861,7 +861,7 @@ jobs:
token: ${{ steps.app-token.outputs.token }}

- name: Setup Just
uses: taiki-e/install-action@9bcaee1dcae34154180f412e2fa69355a7cda9f6 # v2
uses: taiki-e/install-action@16b05812d776ae1dfaabc8277e421fb6d2506419 # v2
with:
tool: just

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/track-next-junctions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
token: ${{ steps.app-token.outputs.token }}

- name: Setup Just
uses: taiki-e/install-action@9bcaee1dcae34154180f412e2fa69355a7cda9f6 # v2
uses: taiki-e/install-action@16b05812d776ae1dfaabc8277e421fb6d2506419 # v2
with:
tool: just

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
uses: ./.github/actions/check-bst2-pin

- name: Setup Just
uses: taiki-e/install-action@9bcaee1dcae34154180f412e2fa69355a7cda9f6 # v2
uses: taiki-e/install-action@16b05812d776ae1dfaabc8277e421fb6d2506419 # v2
with:
tool: just

Expand Down
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ dakota (next→:next/:btw, rolling nightly, no stable promotion)
```

Each image repo pulls `ghcr.io/projectbluefin/common:latest` as a base layer.
testsuite gates `:testing` promotion nightly and `:stable` promotion weekly.
testsuite gates PR changes; stable promotion for Dakota is daily and automated (SHA freshness + cosign verify + boot-check).

**Dakota image streams:**
- `:testing` — `testing` branch, publishes on every BST-changing push (GHA-only changes filtered)
- `:stable` — `main` branch, promoted from `testing` weekly via e2e-gated squash PR
- `:stable` — `main` (bookmark), promoted daily from `:testing` via `execute-release.yml` — no PR, no human approval
- `:next` / `:btw` — `next` branch, GNOME 51 master, fully automated rolling nightly, **no promotion to stable ever**

**`elements/bluefin/common.bst` strips bluefin-only content from common.** Any file added to `common/system_files/shared/` that does not apply to a fresh dakota install must be explicitly `rm -f`'d in the `install-commands` block of that element. Current stripped files: `rechunker-group-fix` script, service, and preset (chunka migration aid — not needed on fresh dakota).
Expand Down Expand Up @@ -210,7 +210,7 @@ Do not request review without evidence. Before opening a PR for review:

**Agents MUST NOT push directly to `main`.** All changes via PR from a feature branch. Branch protection enforces this.

**Dakota promotion PR has no e2e gate by design.** `promote-testing-to-main.yml` passes `run_e2e: false` to `reusable-promote-squash.yml` — the promotion PR gets cosign verification only. Do not add `run_e2e: true` to the promote caller. Promotion from `testing` to `main` is fully automated — no human approval required.
**Dakota stable promotion has no e2e gate by design.** `execute-release.yml` runs SHA freshness check + cosign verify + boot-check, then copies `:testing` → `:stable` directly — no PR, no human approval required. Do not add an e2e gate to the release path.

**Promotion pipeline — cosign verify pattern:** When adding cosign verification to a promotion workflow, anchor the `--certificate-identity-regexp` with `^...$` and restrict it to the specific publishing workflow file and allowed ref patterns (e.g. `^https://github.com/<repo>/.github/workflows/publish\.yml@refs/heads/(main|gh-readonly-queue/main/.+)$`). An unanchored wildcard accepts signatures from any workflow in the repo.

Expand Down
9 changes: 5 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,24 @@ Check the [Contributing Guide](https://docs.projectbluefin.io/contributing) for

## Prerequisites

- [`bst`](https://buildstream.build/) (BuildStream 2) — `pip install buildstream`
- `podman` — required for BST sandbox execution
- `just` — `brew install just` or your OS package manager

BST itself runs inside the pinned `bst2` container — use `just bst ...` rather than installing `bst` directly.

## Pull requests

- Open PRs against the `testing` branch
- Run `just check` before opening a PR
- Run `just validate && just lint` before opening a PR
- Follow [Conventional Commits](https://www.conventionalcommits.org/) for commit messages

## Where things live

```
project.conf # BST project configuration
elements/ # BST element files (.bst) — one per package
junctions/ # BST junction manifests (upstream source pins)
system_files/ # Files overlaid into the final OCI image
patches/ # Patches applied to junction elements via patch_queue
files/ # Static files installed by elements
```

Full build reference and BST workflow: [`AGENTS.md`](AGENTS.md)
16 changes: 3 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,26 +57,16 @@ Leave a comment, challenge the design, propose alternatives. When a discussion r

**Engineers** — these issues have clear acceptance criteria and no open design questions. Pick one up and build it:

### [Agent-ready build queue &rarr;](https://github.com/projectbluefin/dakota/issues?q=is%3Aopen+label%3Aqueue%2Fagent-ready+no%3Aassignee)
### [Agent-ready build queue &rarr;](https://github.com/projectbluefin/dakota/issues?q=is%3Aopen+label%3Astatus%2Fqueued+no%3Aassignee)

Comment `/claim` to take an issue. See [AGENTS.md](AGENTS.md) for the full contributor workflow.

## Help shape what gets built

These issues need human judgment before any code is written — design review, domain knowledge, or hardware context the team doesn't have yet:

### [Issues open for discussion &rarr;](https://github.com/projectbluefin/dakota/issues?q=is%3Aopen+label%3Astatus%2Fdiscussing)

Leave a comment, push back on the design, or share how your hardware is affected. When a discussion reaches consensus, a maintainer marks it `status/approved` and it enters the contributor queue.

Ready to build something? See the [agent-ready queue](https://github.com/projectbluefin/dakota/issues?q=is%3Aopen+label%3Aqueue%2Fagent-ready+no%3Aassignee) for issues with clear acceptance criteria and no open questions.

## Image streams

| Tag | Stream | What it is |
|---|---|---|
| `:stable` | Stable | GNOME 50 — production. Weekly promotion from `:testing`. |
| `:testing` | Dev | GNOME 50 — daily builds from `main`. Gated by e2e before promotion. |
| `:stable` | Stable | GNOME 50 — production. Daily automated promotion from `:testing`. |
| `:testing` | Dev | GNOME 50 — daily builds from `testing` branch. Boot-check gated. |
| `:next` | Rolling | **GNOME master — the bleeding edge.** Tracks gnome-build-meta `master` daily. Auto-updates, zero maintenance. |
| `:btw` | Rolling | Alias for `:next`. |

Expand Down
1 change: 1 addition & 0 deletions docs/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ If your first draft says "use dnf", "edit the Containerfile", or "enable a COPR"
| Package a Go/Rust/Zig/binary/extension project | `docs/skills/packaging-*.md` |
| Test OTA updates locally or on hardware | `docs/skills/local-ota.md` |
| Identify which CI workflow owns the problem | `docs/skills/workflow-map.md` |
| **Any CI failure — load first** | **`docs/skills/ci.md`** |
| Fix reusable workflow / token / cache / startup failures | `docs/skills/ci-tooling.md` |
| Change boot-check, smoke, testsuite, or QEMU CI | `docs/skills/e2e-ci.md` |
| Change promotion PR or stable release flow | `docs/skills/release-promotion.md` |
Expand Down
2 changes: 1 addition & 1 deletion docs/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,6 @@ just bst show --deps all oci/bluefin.bst # full dependency graph
| `rpm-ostree`, `pip install`, `apt-get` in elements | BST-only build; all deps from junctions |
| `$(date)`, `$(hostname)`, `$(curl ...)` in `install-commands` | Breaks reproducibility and BST caching |
| Patch junction files directly | Use `patch_queue` source in the junction `.bst` |
| Force-push to `main` | The merge queue owns merges |
| Force-push to `main` | `main` is a release bookmark; `execute-release.yml` is the only writer |
| Close issues via API or comment | Use `Closes #NNN` in the PR body |
| Open a PR without running `just validate` | Wastes everyone's time |
69 changes: 31 additions & 38 deletions docs/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,52 +6,52 @@
|---|---|---|
| `validate` | `pull_request` | `bst show` — graph + patch check (~15 min) |
| `e2e` | `pull_request` when `elements/`, `files/`, `patches/`, `Justfile`, or `project.conf` changed | Smoke test in QEMU via projectbluefin/testsuite |
| `build` | `merge_group`, `workflow_dispatch`, `schedule` — skips on `pull_request` | Full OCI build (~60–90 min) |
| `build-aarch64` | disabled | ARM64 — pending investigation |
| `build` | `push: testing/next` (paths-ignore: `.github/workflows/**`, `docs/**`, `**.md`, `AGENTS.md`), `merge_group`, `workflow_dispatch`, `schedule: daily 13:00 UTC` — skips on `pull_request` | Full OCI build (~60–90 min) |
| `build-aarch64` | `push: testing/main` (BST-affecting paths only), `workflow_run` from `publish.yml` on `testing`, `workflow_dispatch` | ARM64 — fully decoupled, never blocks release |

## Publish pipeline (publish.yml)

`build` success on main/testing/next triggers publish.yml via `workflow_run`:
`build` success on `testing` or `next` triggers `publish.yml` via `workflow_run`:

```
build.yml (main|testing|next) → [workflow_run] → publish.yml
setup → publish-image (matrix) → promote (:testing or :next)
└──────────────→ publish-sbom
build.yml (testing|next) → [workflow_run] → publish.yml
setup → publish-image → boot-check → promote (:testing or :next)
└──────────→ publish-sbom (parallel)
```

| Job | What |
|---|---|
| `setup` | Resolves SHA, trigger event, and branch |
| `publish-image` | Exports from CAS; runs `chunka@v1` to rechunk; pushes `:$sha`; signs + attests |
| `promote` | `skopeo copy` `:$sha` → `:testing` (merge-queue/schedule/dispatch) |
| `boot-check` | Hard gate — image must boot before `:testing` is promoted |
| `promote` | `skopeo copy` `:$sha` → `:testing` (only runs after boot-check passes) |
| `publish-sbom` | Generates SBOM; attaches via oras; signs SBOM (runs in parallel with promote) |

`promote` depends only on `publish-image`, not on SBOM — saves 10–15 min on the critical path.

**`execute-release.yml`** fires on `push: main` and `workflow_dispatch`. A `check-trigger` job reads the commit message — proceeds only when it matches `^ci\(promote\): dakota testing` or `^chore: promote testing to main`. `workflow_dispatch` bypasses the gate. On success: copies `:testing` → `:stable`, then generates a GitHub Release with SBOM diff.
`promote` depends only on `publish-image` + `boot-check`, not on SBOM — saves 10–15 min on the critical path.

**Critical ordering:** `publish.yml` pulls the OCI artifact from CAS. The artifact is only in CAS if `build.yml` ran first for that SHA. Always dispatch `build.yml --ref testing` (or let push trigger it) before manually dispatching `publish.yml`.

## Stable promotion (execute-release.yml)

Triggered by a push to `main` whose commit message matches the promotion pattern. The normal path is:
`execute-release.yml` fires via `workflow_run` from `publish.yml` on the `testing` branch — no commit message gate, no PR, no human approval.

```
push to testing (BST-affecting)
→ build.yml → publish.yml → :testing
→ promote-testing-to-main.yml → auto/promote-testing-to-main PR
→ pr-release-gate.yml (cosign verify)
→ auto-merge → push to main (commit: "ci(promote): dakota testing ...")
→ execute-release.yml (check-trigger passes)
→ :testing copied to :stable
→ GitHub Release created
push to testing (BST-affecting) or daily 13:00 UTC schedule
→ build.yml → publish.yml → boot-check → :testing
→ execute-release.yml (workflow_run from publish on testing)
→ SHA freshness check (:testing SHA vs :stable SHA)
→ skip if equal (already up to date)
→ cosign verify :testing
→ skopeo copy :testing → :stable
→ fast-forward main bookmark
→ GitHub Release created
```

Schedule: `promote-testing-to-main.yml` runs `cron: '0 4 * * 2'` (Tuesday 04:00 UTC). That is the only automated promotion cadence.
`main` is a **release bookmark only** — fast-forwarded by `execute-release.yml` after each successful promotion. Do not open PRs against `main`.

## Schedule

Builds fire on schedule (13:00 UTC for testing, 03:00 UTC for next), merge_group, or workflow_dispatch.
Build fires daily at 13:00 UTC (`schedule:` in `build.yml`), plus on every BST-affecting push to `testing` or `next`, `merge_group`, and `workflow_dispatch`.

## Remote cache

Expand All @@ -62,8 +62,9 @@ Builds fire on schedule (13:00 UTC for testing, 03:00 UTC for next), merge_group
`ghcr.io/projectbluefin/dakota:{testing,stable,next,btw}` and `ghcr.io/projectbluefin/dakota:<sha>`

Streams:
- `:testing` — published on every BST-affecting push to `testing` or `main` branch
- `:stable` — promoted from `:testing` via `execute-release.yml` after promotion PR merges to main (Tuesday 04:00 UTC scheduled path, or manual dispatch)
- `:testing` — published on every BST-affecting push to the `testing` branch (or daily schedule)
- `:stable` — promoted from `:testing` daily by `execute-release.yml` (when `:testing` SHA differs from `:stable`)
- `:next` / `:btw` — published from the `next` branch; never promoted to `:stable`

Never bypass the merge queue with `--admin`.

Expand All @@ -72,14 +73,8 @@ Never bypass the merge queue with `--admin`.
To manually cut a `:stable` release:

```bash
# 1. Ensure :testing exists and promotion PR is open
gh pr list --repo projectbluefin/dakota --search 'head:auto/promote-testing-to-main state:open'

# 2. If the promotion PR gate has passed, dispatch execute-release directly
gh workflow run execute-release.yml --repo projectbluefin/dakota --ref main

# OR: dispatch promote-testing-to-main to open/update the promotion PR
gh workflow run promote-testing-to-main.yml --repo projectbluefin/dakota
# 1. Verify :testing is fresh and cosign-verified, then dispatch execute-release directly
gh workflow run execute-release.yml --repo projectbluefin/dakota --ref testing
```

## Restarting the factory (publish pipeline has been idle)
Expand All @@ -91,17 +86,15 @@ the restart sequence is:
# 1. Verify publish.yml is healthy — no startup_failure
gh run list --repo projectbluefin/dakota --workflow publish.yml --limit 5

# 2. Dispatch a fresh build on main to populate the CAS
gh workflow run build.yml --repo projectbluefin/dakota --ref main
# 2. Dispatch a fresh build on testing to populate the CAS
gh workflow run build.yml --repo projectbluefin/dakota --ref testing
# Wait ~60–90 minutes for build to complete

# 3. Dispatch publish.yml after build finishes (or let workflow_run auto-trigger)
gh workflow run publish.yml --repo projectbluefin/dakota --ref main
# 3. publish.yml auto-triggers via workflow_run; if not, dispatch manually
gh workflow run publish.yml --repo projectbluefin/dakota --ref testing

# 4. Monitor until :testing lands
# 4. Monitor until :testing lands, then execute-release auto-triggers
gh run watch --repo projectbluefin/dakota

# 5. Cut stable release (see Manual stable promotion above)
```

**Common failure: `startup_failure` with `jobs: []`**
Expand Down
Loading