|
| 1 | +# AGENTS.md |
| 2 | + |
| 3 | +Canonical contract for `safelibs/port-template` and every `safelibs/port-*` repository created from it. Read this before adding a new port, modifying a script under `scripts/`, or editing the workflow file. |
| 4 | + |
| 5 | +## Why This Document Exists |
| 6 | + |
| 7 | +Earlier versions of the template defined CI as a fixed sequence of inline workflow steps. Each port that diverged from the default needed a per-port workflow override, and a separate generator in `safelibs/apt` rendered those overrides from a central config. That bifurcation made it brittle to evolve either side. |
| 8 | + |
| 9 | +The current shape moves the divergence into hook scripts under `scripts/`. The workflow runs a fixed sequence of those hooks. Ports own their own hook contents; the template ships sensible defaults. There is no generator and no central config — each port is self-describing. |
| 10 | + |
| 11 | +## CI Pipeline |
| 12 | + |
| 13 | +`.github/workflows/ci-release.yml` runs the following sequence on every push to `main` and every manual `workflow_dispatch`: |
| 14 | + |
| 15 | +| # | Step | Script | Template default | |
| 16 | +| - | ---- | ------ | ---------------- | |
| 17 | +| 1 | Install build dependencies | `scripts/install-build-deps.sh` | No-op | |
| 18 | +| 2 | Check repository layout | `scripts/check-layout.sh` | Lint required files / executable bits | |
| 19 | +| 3 | Build .deb artifacts | `scripts/build-debs.sh` | Reference build from `packaging/package.env` + `safe/` | |
| 20 | +| 4 | Run upstream tests | `scripts/run-upstream-tests.sh` | Run every `*.sh` under `tests/upstream/` | |
| 21 | +| 5 | Run port tests | `scripts/run-port-tests.sh` | Run every `*.sh` under `tests/port/` | |
| 22 | +| 6 | Run validation tests | `scripts/run-validation-tests.sh` | Clone `safelibs/validator`, run `port-04-test` mode against `dist/*.deb` | |
| 23 | +| 7 | Upload `dist/*.deb` | (workflow) | One GitHub Actions artifact per run | |
| 24 | +| 8 | Publish release | (workflow) | `build-<short-sha>` GitHub Release with every `dist/*.deb` | |
| 25 | + |
| 26 | +Steps 1–6 are hooks. Steps 7–8 are workflow-owned and ports do not customize them. |
| 27 | + |
| 28 | +## Script Contracts |
| 29 | + |
| 30 | +Each hook script is invoked with `bash <script>` from the repository root. `set -euo pipefail` is the convention. Scripts must succeed (exit 0) on the happy path; non-zero is a CI failure. |
| 31 | + |
| 32 | +### `scripts/install-build-deps.sh` |
| 33 | + |
| 34 | +Install everything the build and tests need that is not preinstalled on `ubuntu-latest`: |
| 35 | + |
| 36 | +- apt packages (compilers, dev libraries, packaging tools) |
| 37 | +- language toolchains (rustup, cargo, custom Python venvs) |
| 38 | +- any other system-level setup |
| 39 | + |
| 40 | +May invoke `sudo`. Must be idempotent — reruns on a warm runner must succeed. Template default is a no-op because the reference build only needs `dpkg-deb`, which is preinstalled. |
| 41 | + |
| 42 | +### `scripts/check-layout.sh` |
| 43 | + |
| 44 | +Lint the repository against the template contract: required files exist, scripts are executable, JSON inventories parse, `.gitattributes` carries the expected entries, and `packaging/package.env` is well-formed. A port can extend this with port-specific invariants but must keep the baseline checks. |
| 45 | + |
| 46 | +### `scripts/build-debs.sh` |
| 47 | + |
| 48 | +Produce one or more Debian package files under `dist/`. There is no upper bound on the number of `.deb` files — most ports emit a handful (runtime, dev, tools, docs). |
| 49 | + |
| 50 | +The reference implementation copies `safe/` into `DEB_INSTALL_PREFIX` from `packaging/package.env` and emits a single `.deb` via `dpkg-deb --build`. Real ports usually replace this with `dpkg-buildpackage -us -uc -b` rooted in `safe/debian/`, or with a port-owned build script (`bash safe/scripts/build-deb.sh`, `cargo run -p xtask -- package-deb`, etc.). |
| 51 | + |
| 52 | +`SAFELIBS_COMMIT_SHA` is set in CI; a build script that stamps versions should consume it via that variable. |
| 53 | + |
| 54 | +### `scripts/run-upstream-tests.sh` |
| 55 | + |
| 56 | +Run the upstream library's regression suite against the just-built safe `.deb`s. The name was chosen to be unambiguous: it runs *upstream's* suite, not tests of upstream's behavior. The intent is to prove the safe implementation is API/ABI-compatible with what real consumers of the upstream library expect. |
| 57 | + |
| 58 | +The template default scans `tests/upstream/*.sh`. A real port typically replaces this with a script that installs `dist/*.deb` into a chroot or container and invokes the upstream test harness via `make check`, `meson test`, or similar. |
| 59 | + |
| 60 | +This script may need the artifacts produced by `build-debs.sh`. CI runs it after the build; local invocations must run the build first. |
| 61 | + |
| 62 | +### `scripts/run-port-tests.sh` |
| 63 | + |
| 64 | +Run port-authored tests for the safe implementation: unit tests, ABI checks, fuzzing harnesses, differential tests against upstream. The template default scans `tests/port/*.sh`. Ports replace it with whatever framework fits their language and toolchain. |
| 65 | + |
| 66 | +### `scripts/run-validation-tests.sh` |
| 67 | + |
| 68 | +Run the [safelibs/validator](https://github.com/safelibs/validator) test matrix in `port-04-test` mode against `dist/*.deb`. |
| 69 | + |
| 70 | +Inputs (mostly read from `packaging/package.env`): |
| 71 | + |
| 72 | +- `SAFELIBS_LIBRARY` — must match a `name:` entry in the validator's `repositories.yml`. |
| 73 | +- `dist/*.deb` — produced by the build hook. |
| 74 | +- `SAFELIBS_COMMIT_SHA` — used as the synthetic release tag. |
| 75 | + |
| 76 | +Optional environment overrides: |
| 77 | + |
| 78 | +- `SAFELIBS_VALIDATOR_DIR` — path to an existing validator checkout. When unset, the script clones `https://github.com/safelibs/validator` into `.work/validator`. |
| 79 | +- `SAFELIBS_VALIDATOR_REF` — git ref to clone (default `main`). |
| 80 | +- `SAFELIBS_VALIDATOR_REPO` — git remote (default `https://github.com/safelibs/validator`). |
| 81 | +- `SAFELIBS_RECORD_CASTS` — non-empty enables `--record-casts`. |
| 82 | + |
| 83 | +Behavior: |
| 84 | + |
| 85 | +1. Reads canonical `apt_packages` for `SAFELIBS_LIBRARY` from the validator manifest. |
| 86 | +2. Inspects every `dist/*.deb`, matching them by `dpkg-deb --field Package` against the canonical list. Non-canonical extras are ignored. Canonical packages with no matching deb become `unported_original_packages`. |
| 87 | +3. Synthesizes a `port-04-test` deb lock JSON file with the matching debs, sha256s, sizes, and the synthesized `release_tag = build-<commit[:12]>`. |
| 88 | +4. Lays out `<override-deb-root>/<library>/<filename>.deb`. |
| 89 | +5. Invokes `bash <validator>/test.sh --library <SAFELIBS_LIBRARY> --mode port-04-test --override-deb-root ... --port-deb-lock ... --artifact-root ...`. |
| 90 | + |
| 91 | +Soft skip: a library that has no entry in the validator manifest (the template itself, ports still being authored) returns a skip and the script exits 0. This is the only acceptable success without a real validator run. |
| 92 | + |
| 93 | +Hard failures: missing `dist/*.deb`, no canonical packages matched, mismatch between dist debs and canonical packages, validator clone failure, validator matrix failure. |
| 94 | + |
| 95 | +## Repository Layout |
| 96 | + |
| 97 | +Required directories: `original/`, `safe/`, `packaging/`, `docs/`, `tests/upstream/`, `tests/port/`, `scripts/`. |
| 98 | + |
| 99 | +Required files: `.github/workflows/ci-release.yml`, `.gitattributes`, `README.md`, `AGENTS.md`, `CLAUDE.md`, `all_cves.json`, `dependents.json`, `relevant_cves.json`, `docs/PORTING.md`, `docs/PUBLISHING.md`, `packaging/package.env`. |
| 100 | + |
| 101 | +Required executable scripts: `scripts/build-debs.sh`, `scripts/check-layout.sh`, `scripts/install-build-deps.sh`, `scripts/run-port-tests.sh`, `scripts/run-tests.sh`, `scripts/run-upstream-tests.sh`, `scripts/run-validation-tests.sh`. |
| 102 | + |
| 103 | +`scripts/check-layout.sh` is the source of truth for the layout. When you change required files or scripts, update it in the same commit. |
| 104 | + |
| 105 | +## `packaging/package.env` |
| 106 | + |
| 107 | +The reference build script consumes this file. Required fields: |
| 108 | + |
| 109 | +- `SAFELIBS_LIBRARY` — validator manifest identifier and `safelibs/port-<library>` suffix. |
| 110 | +- `DEB_PACKAGE`, `DEB_VERSION`, `DEB_ARCHITECTURE`, `DEB_MAINTAINER`, `DEB_SECTION`, `DEB_PRIORITY`, `DEB_DESCRIPTION`, `DEB_INSTALL_PREFIX`, `DEB_DEPENDS` — Debian package metadata. |
| 111 | + |
| 112 | +Ports that override `build-debs.sh` may leave the `DEB_*` fields at their template defaults but must still set a real `SAFELIBS_LIBRARY` for the validator hook. |
| 113 | + |
| 114 | +## When To Edit What |
| 115 | + |
| 116 | +- **Adding/changing a port-specific build step:** edit the relevant hook script in your port repo. Do not edit the workflow file. |
| 117 | +- **Changing the workflow shape (new step order, new global env, new artifact policy):** edit the template's `.github/workflows/ci-release.yml` and the corresponding section in this file. Sync changes back into `safelibs/port-*` repos by hand or by re-templating. |
| 118 | +- **Adding a new required file or directory:** update `scripts/check-layout.sh`, this file, and `README.md` in the same commit. |
| 119 | +- **Renaming a hook script:** update `scripts/check-layout.sh`, `.github/workflows/ci-release.yml`, this file, `README.md`, `docs/PORTING.md`, and `docs/PUBLISHING.md` in the same commit. |
| 120 | + |
| 121 | +## Non-Goals |
| 122 | + |
| 123 | +- The template is **not** a generator. There is no central config that produces per-port workflows. Every port owns its scripts. |
| 124 | +- The template is **not** a runtime library. It only defines layout and CI shape. |
| 125 | +- The template does **not** support cross-port dependencies. A port hook may not assume another port has already run. |
0 commit comments