Skip to content

Commit 42f02e1

Browse files
zardusclaude
andcommitted
Adopt safelibs/port-template hook-script CI contract
Replace the auto-generated build-debs.yml with the hook-script contract from safelibs/port-template. The workflow runs a fixed sequence under scripts/: install-build-deps, check-layout, build-debs, run-upstream-tests, run-port-tests, run-validation-tests; uploads dist/*.deb; publishes a build-<short-sha> GitHub release. scripts/install-build-deps.sh and scripts/build-debs.sh are port-specific and translate this port's entry from apt/repositories.yml port_build_overrides into bash. The remaining hooks are template defaults. See AGENTS.md for the full contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 3172ab2 commit 42f02e1

20 files changed

Lines changed: 1058 additions & 121 deletions

.gitattributes

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
*.sh text eol=lf
2+
*.json text eol=lf
3+
*.md text eol=lf
4+
*.env text eol=lf
5+
.github/workflows/*.yml text eol=lf

.github/workflows/build-debs.yml

Lines changed: 0 additions & 121 deletions
This file was deleted.

.github/workflows/ci-release.yml

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
name: ci-release
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: write
11+
12+
defaults:
13+
run:
14+
shell: bash
15+
16+
jobs:
17+
test-build-release:
18+
runs-on: ubuntu-latest
19+
steps:
20+
- name: Check out repository
21+
uses: actions/checkout@v4
22+
with:
23+
fetch-depth: 0
24+
25+
- name: Print tool versions
26+
run: |
27+
set -euo pipefail
28+
bash --version
29+
git --version
30+
python3 --version
31+
dpkg --version
32+
dpkg-deb --version
33+
34+
- name: Install build dependencies
35+
run: bash scripts/install-build-deps.sh
36+
37+
- name: Check repository layout
38+
run: bash scripts/check-layout.sh
39+
40+
- name: Build .deb artifacts
41+
env:
42+
SAFELIBS_COMMIT_SHA: ${{ github.sha }}
43+
run: |
44+
set -euo pipefail
45+
rm -rf build dist
46+
bash scripts/build-debs.sh
47+
48+
- name: Run upstream tests
49+
env:
50+
SAFELIBS_COMMIT_SHA: ${{ github.sha }}
51+
run: bash scripts/run-upstream-tests.sh
52+
53+
- name: Run port tests
54+
env:
55+
SAFELIBS_COMMIT_SHA: ${{ github.sha }}
56+
run: bash scripts/run-port-tests.sh
57+
58+
- name: Run validation tests
59+
env:
60+
SAFELIBS_COMMIT_SHA: ${{ github.sha }}
61+
run: bash scripts/run-validation-tests.sh
62+
63+
- name: Collect built artifacts
64+
id: artifact
65+
run: |
66+
set -euo pipefail
67+
shopt -s nullglob
68+
mapfile -t debs < <(find dist -type f -name '*.deb' -print | sort)
69+
if (( ${#debs[@]} == 0 )); then
70+
printf 'No .deb files were produced under dist/.\n' >&2
71+
exit 1
72+
fi
73+
{
74+
printf 'count=%d\n' "${#debs[@]}"
75+
printf 'paths<<EOF\n'
76+
printf '%s\n' "${debs[@]}"
77+
printf 'EOF\n'
78+
} >> "$GITHUB_OUTPUT"
79+
printf 'Built %d .deb file(s):\n' "${#debs[@]}"
80+
printf ' %s\n' "${debs[@]}"
81+
82+
- name: Upload Debian artifacts
83+
uses: actions/upload-artifact@v4
84+
with:
85+
name: debs-${{ github.run_id }}
86+
path: dist/*.deb
87+
if-no-files-found: error
88+
89+
- name: Publish build release
90+
if: github.event_name == 'push'
91+
env:
92+
GH_TOKEN: ${{ github.token }}
93+
DEB_PATHS: ${{ steps.artifact.outputs.paths }}
94+
run: |
95+
set -euo pipefail
96+
97+
short_sha="${GITHUB_SHA:0:12}"
98+
tag="build-${short_sha}"
99+
100+
mapfile -t deb_files <<< "$DEB_PATHS"
101+
102+
notes_file="$(mktemp)"
103+
{
104+
printf 'Commit SHA: %s\n\n' "$GITHUB_SHA"
105+
printf 'Attached .deb file(s) were produced by `scripts/build-debs.sh`.\n'
106+
} > "$notes_file"
107+
108+
if gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
109+
gh release upload "$tag" "${deb_files[@]}" --repo "$GITHUB_REPOSITORY" --clobber
110+
else
111+
gh release create "$tag" "${deb_files[@]}" \
112+
--repo "$GITHUB_REPOSITORY" \
113+
--target "$GITHUB_SHA" \
114+
--title "Safe library debs for ${short_sha}" \
115+
--notes-file "$notes_file"
116+
fi
117+
118+
rm -f "$notes_file"

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/.work/
2+
/build/
3+
/dist/
4+
*.deb
5+
*.ddeb
6+
*.buildinfo
7+
*.changes
8+
*.log

AGENTS.md

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# AGENTS.md
2+
3+
Canonical contract for `safelibs/port-template` and every `safelibs/port-*` repository created from it. Read this before adding a new port, modifying a script under `scripts/`, or editing the workflow file.
4+
5+
## Why This Document Exists
6+
7+
Earlier versions of the template defined CI as a fixed sequence of inline workflow steps. Each port that diverged from the default needed a per-port workflow override, and a separate generator in `safelibs/apt` rendered those overrides from a central config. That bifurcation made it brittle to evolve either side.
8+
9+
The current shape moves the divergence into hook scripts under `scripts/`. The workflow runs a fixed sequence of those hooks. Ports own their own hook contents; the template ships sensible defaults. There is no generator and no central config — each port is self-describing.
10+
11+
## CI Pipeline
12+
13+
`.github/workflows/ci-release.yml` runs the following sequence on every push to `main` and every manual `workflow_dispatch`:
14+
15+
| # | Step | Script | Template default |
16+
| - | ---- | ------ | ---------------- |
17+
| 1 | Install build dependencies | `scripts/install-build-deps.sh` | No-op |
18+
| 2 | Check repository layout | `scripts/check-layout.sh` | Lint required files / executable bits |
19+
| 3 | Build .deb artifacts | `scripts/build-debs.sh` | Reference build from `packaging/package.env` + `safe/` |
20+
| 4 | Run upstream tests | `scripts/run-upstream-tests.sh` | Run every `*.sh` under `tests/upstream/` |
21+
| 5 | Run port tests | `scripts/run-port-tests.sh` | Run every `*.sh` under `tests/port/` |
22+
| 6 | Run validation tests | `scripts/run-validation-tests.sh` | Clone `safelibs/validator`, run `port-04-test` mode against `dist/*.deb` |
23+
| 7 | Upload `dist/*.deb` | (workflow) | One GitHub Actions artifact per run |
24+
| 8 | Publish release | (workflow) | `build-<short-sha>` GitHub Release with every `dist/*.deb` |
25+
26+
Steps 1–6 are hooks. Steps 7–8 are workflow-owned and ports do not customize them.
27+
28+
## Script Contracts
29+
30+
Each hook script is invoked with `bash <script>` from the repository root. `set -euo pipefail` is the convention. Scripts must succeed (exit 0) on the happy path; non-zero is a CI failure.
31+
32+
### `scripts/install-build-deps.sh`
33+
34+
Install everything the build and tests need that is not preinstalled on `ubuntu-latest`:
35+
36+
- apt packages (compilers, dev libraries, packaging tools)
37+
- language toolchains (rustup, cargo, custom Python venvs)
38+
- any other system-level setup
39+
40+
May invoke `sudo`. Must be idempotent — reruns on a warm runner must succeed. Template default is a no-op because the reference build only needs `dpkg-deb`, which is preinstalled.
41+
42+
### `scripts/check-layout.sh`
43+
44+
Lint the repository against the template contract: required files exist, scripts are executable, JSON inventories parse, `.gitattributes` carries the expected entries, and `packaging/package.env` is well-formed. A port can extend this with port-specific invariants but must keep the baseline checks.
45+
46+
### `scripts/build-debs.sh`
47+
48+
Produce one or more Debian package files under `dist/`. There is no upper bound on the number of `.deb` files — most ports emit a handful (runtime, dev, tools, docs).
49+
50+
The reference implementation copies `safe/` into `DEB_INSTALL_PREFIX` from `packaging/package.env` and emits a single `.deb` via `dpkg-deb --build`. Real ports usually replace this with `dpkg-buildpackage -us -uc -b` rooted in `safe/debian/`, or with a port-owned build script (`bash safe/scripts/build-deb.sh`, `cargo run -p xtask -- package-deb`, etc.).
51+
52+
`SAFELIBS_COMMIT_SHA` is set in CI; a build script that stamps versions should consume it via that variable.
53+
54+
### `scripts/run-upstream-tests.sh`
55+
56+
Run the upstream library's regression suite against the just-built safe `.deb`s. The name was chosen to be unambiguous: it runs *upstream's* suite, not tests of upstream's behavior. The intent is to prove the safe implementation is API/ABI-compatible with what real consumers of the upstream library expect.
57+
58+
The template default scans `tests/upstream/*.sh`. A real port typically replaces this with a script that installs `dist/*.deb` into a chroot or container and invokes the upstream test harness via `make check`, `meson test`, or similar.
59+
60+
This script may need the artifacts produced by `build-debs.sh`. CI runs it after the build; local invocations must run the build first.
61+
62+
### `scripts/run-port-tests.sh`
63+
64+
Run port-authored tests for the safe implementation: unit tests, ABI checks, fuzzing harnesses, differential tests against upstream. The template default scans `tests/port/*.sh`. Ports replace it with whatever framework fits their language and toolchain.
65+
66+
### `scripts/run-validation-tests.sh`
67+
68+
Run the [safelibs/validator](https://github.com/safelibs/validator) test matrix in `port-04-test` mode against `dist/*.deb`.
69+
70+
Inputs (mostly read from `packaging/package.env`):
71+
72+
- `SAFELIBS_LIBRARY` — must match a `name:` entry in the validator's `repositories.yml`.
73+
- `dist/*.deb` — produced by the build hook.
74+
- `SAFELIBS_COMMIT_SHA` — used as the synthetic release tag.
75+
76+
Optional environment overrides:
77+
78+
- `SAFELIBS_VALIDATOR_DIR` — path to an existing validator checkout. When unset, the script clones `https://github.com/safelibs/validator` into `.work/validator`.
79+
- `SAFELIBS_VALIDATOR_REF` — git ref to clone (default `main`).
80+
- `SAFELIBS_VALIDATOR_REPO` — git remote (default `https://github.com/safelibs/validator`).
81+
- `SAFELIBS_RECORD_CASTS` — non-empty enables `--record-casts`.
82+
83+
Behavior:
84+
85+
1. Reads canonical `apt_packages` for `SAFELIBS_LIBRARY` from the validator manifest.
86+
2. Inspects every `dist/*.deb`, matching them by `dpkg-deb --field Package` against the canonical list. Non-canonical extras are ignored. Canonical packages with no matching deb become `unported_original_packages`.
87+
3. Synthesizes a `port-04-test` deb lock JSON file with the matching debs, sha256s, sizes, and the synthesized `release_tag = build-<commit[:12]>`.
88+
4. Lays out `<override-deb-root>/<library>/<filename>.deb`.
89+
5. Invokes `bash <validator>/test.sh --library <SAFELIBS_LIBRARY> --mode port-04-test --override-deb-root ... --port-deb-lock ... --artifact-root ...`.
90+
91+
Soft skip: a library that has no entry in the validator manifest (the template itself, ports still being authored) returns a skip and the script exits 0. This is the only acceptable success without a real validator run.
92+
93+
Hard failures: missing `dist/*.deb`, no canonical packages matched, mismatch between dist debs and canonical packages, validator clone failure, validator matrix failure.
94+
95+
## Repository Layout
96+
97+
Required directories: `original/`, `safe/`, `packaging/`, `docs/`, `tests/upstream/`, `tests/port/`, `scripts/`.
98+
99+
Required files: `.github/workflows/ci-release.yml`, `.gitattributes`, `README.md`, `AGENTS.md`, `CLAUDE.md`, `all_cves.json`, `dependents.json`, `relevant_cves.json`, `docs/PORTING.md`, `docs/PUBLISHING.md`, `packaging/package.env`.
100+
101+
Required executable scripts: `scripts/build-debs.sh`, `scripts/check-layout.sh`, `scripts/install-build-deps.sh`, `scripts/run-port-tests.sh`, `scripts/run-tests.sh`, `scripts/run-upstream-tests.sh`, `scripts/run-validation-tests.sh`.
102+
103+
`scripts/check-layout.sh` is the source of truth for the layout. When you change required files or scripts, update it in the same commit.
104+
105+
## `packaging/package.env`
106+
107+
The reference build script consumes this file. Required fields:
108+
109+
- `SAFELIBS_LIBRARY` — validator manifest identifier and `safelibs/port-<library>` suffix.
110+
- `DEB_PACKAGE`, `DEB_VERSION`, `DEB_ARCHITECTURE`, `DEB_MAINTAINER`, `DEB_SECTION`, `DEB_PRIORITY`, `DEB_DESCRIPTION`, `DEB_INSTALL_PREFIX`, `DEB_DEPENDS` — Debian package metadata.
111+
112+
Ports that override `build-debs.sh` may leave the `DEB_*` fields at their template defaults but must still set a real `SAFELIBS_LIBRARY` for the validator hook.
113+
114+
## When To Edit What
115+
116+
- **Adding/changing a port-specific build step:** edit the relevant hook script in your port repo. Do not edit the workflow file.
117+
- **Changing the workflow shape (new step order, new global env, new artifact policy):** edit the template's `.github/workflows/ci-release.yml` and the corresponding section in this file. Sync changes back into `safelibs/port-*` repos by hand or by re-templating.
118+
- **Adding a new required file or directory:** update `scripts/check-layout.sh`, this file, and `README.md` in the same commit.
119+
- **Renaming a hook script:** update `scripts/check-layout.sh`, `.github/workflows/ci-release.yml`, this file, `README.md`, `docs/PORTING.md`, and `docs/PUBLISHING.md` in the same commit.
120+
121+
## Non-Goals
122+
123+
- The template is **not** a generator. There is no central config that produces per-port workflows. Every port owns its scripts.
124+
- The template is **not** a runtime library. It only defines layout and CI shape.
125+
- The template does **not** support cross-port dependencies. A port hook may not assume another port has already run.

CLAUDE.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# CLAUDE.md
2+
3+
See [AGENTS.md](AGENTS.md) for the canonical contract that governs this repository: the CI hook sequence, what each `scripts/*.sh` must do, the required layout, and when to edit which files. Both human and agent contributors should treat AGENTS.md as the source of truth.

0 commit comments

Comments
 (0)