diff --git a/.github/workflows/dogfood.yml b/.github/workflows/dogfood.yml new file mode 100644 index 0000000..1485ba6 --- /dev/null +++ b/.github/workflows/dogfood.yml @@ -0,0 +1,39 @@ +name: Dogfooding + +# Proofkeeper verifies itself: run its own `coverage` command against its own +# Lore corpus (lore-proofkeeper/) and fail if any capability is unverified. This +# turns the dogfood signal into an enforced gate — the corpus can never drift +# un-green. It exercises the real CLI and the real `rac export --graph` contract +# path (ADR-063, ADR-083), not a bespoke checker. + +on: + push: + branches: ["**"] + pull_request: + +jobs: + coverage: + name: self-coverage (proofkeeper verifies proofkeeper) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: "20" + cache: npm + - run: npm ci + - run: npm run build + + # The `rac` engine is a contract dependency, not a package dependency: + # install it as an external CLI from rac-core (its distribution is + # "requirements-as-code"; the console script is `rac`). Proofkeeper consumes + # only its published `rac export --graph` JSON output. + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install the rac engine (provides the `rac` CLI) + run: pip install "git+https://github.com/itsthelore/rac-core.git" + + - name: Proofkeeper coverage over its own corpus (fails on any unverified) + run: node dist/cli.js coverage --corpus lore-proofkeeper/ diff --git a/README.md b/README.md index 3e99053..7006b91 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,7 @@ Banner: add docs/assets/proofkeeper-header-{dark,light}.png, then uncomment.

CI +Dogfooding: Proofkeeper verifies its own corpus npm Node >= 20 TypeScript diff --git a/lore-proofkeeper/designs/design-dogfood-coverage-gate.md b/lore-proofkeeper/designs/design-dogfood-coverage-gate.md new file mode 100644 index 0000000..0455cc0 --- /dev/null +++ b/lore-proofkeeper/designs/design-dogfood-coverage-gate.md @@ -0,0 +1,63 @@ +--- +schema_version: 1 +id: PK-KWBH4HMAVW38 +type: design +--- +# Self-Verification Coverage Gate + +## Context + +Proofkeeper records its own capabilities in `lore-proofkeeper/` and proves them with `## Verified By` links, but the dogfood report was never enforced. This design makes "Proofkeeper's corpus stays fully verified" a required CI gate, using Proofkeeper's own coverage command, and surfaces it as a README badge. + +## User Need + +A maintainer (and a reader evaluating the project) needs confidence that Proofkeeper actually practises what it sells — that every capability it claims is backed by a test — without manually running coverage and without trusting an advisory number that can silently drift. + +## Design + +- A dedicated workflow `.github/workflows/dogfood.yml`, named so its status badge reads as the dogfood signal. On push and pull request it: + - checks out the repo, sets up Node, `npm ci`, `npm run build`; + - sets up Python and installs the `rac` engine from the rac-core repository (`pip install "git+https://github.com/itsthelore/rac-core.git"`), which provides the `rac` command; + - runs `node dist/cli.js coverage --corpus lore-proofkeeper/`. The command exits non-zero if any capability is unverified, so the job fails on drift with no extra scripting. +- The README adds a badge pointing at this workflow's status on the default branch (a shields.io GitHub-Actions badge with a `dogfooding` label), next to the existing CI and npm badges. +- This requirement is itself verified by the workflow file, keeping the corpus self-consistent (every capability, including this one, has a `## Verified By`). + +## Constraints + +- Reuse the published behaviour: the gate runs the real `coverage` CLI and the real `rac export` contract path, not a bespoke checker (ADR-063, ADR-083). +- No engine import: `rac` is installed as an external CLI; Proofkeeper consumes its JSON output only. +- Determinism over snapshots: export the corpus live rather than committing a graph file that could drift from the artifacts. + +## Rationale + +Enforcing the existing coverage command is the smallest change that turns an advisory signal into a guarantee, and it dogfoods the product end to end — Proofkeeper verifies itself exactly as a consumer would. A workflow-status badge is honest because it reflects a real run, not a hand-set value. + +## Alternatives + +- **Commit a `rac export --graph` snapshot and check against it.** Rejected: the snapshot drifts from the artifacts and would need its own freshness gate. +- **A custom script that parses the corpus directly.** Rejected: it would bypass the very CLI and contract path the product ships, weakening the dogfood. +- **A manually maintained "passing" badge.** Rejected: not tied to a real result; can lie. + +## Accessibility + +The badge carries descriptive alt text; status is conveyed by text ("passing"/"failing"), not colour alone. + +## Style Guidance + +Match the existing workflow conventions in `.github/workflows/ci.yml` (Node 20, `npm ci`, named jobs) and place the badge alongside the current badge row. + +## Open Questions + +- Whether to pin the rac-core install to a released tag for extra determinism once a stable cadence exists. Deferred; track via the schema_version guard for now. + +## Related Requirements + +- req-dogfood-coverage-gate + +## Related Roadmaps + +- autonomous-qa-enhancements + +## Status + +Accepted diff --git a/lore-proofkeeper/requirements/req-dogfood-coverage-gate.md b/lore-proofkeeper/requirements/req-dogfood-coverage-gate.md new file mode 100644 index 0000000..2b300e3 --- /dev/null +++ b/lore-proofkeeper/requirements/req-dogfood-coverage-gate.md @@ -0,0 +1,40 @@ +--- +schema_version: 1 +id: PK-KWBH4H0WB6VH +type: requirement +--- +# Self-Verification Coverage Gate + +## Problem + +Proofkeeper dogfoods itself: its own capabilities live as requirements in `lore-proofkeeper/`, and `proofkeeper coverage --corpus lore-proofkeeper/` reports which have a verifying test. But nothing enforced that report — a capability could be added without a `## Verified By` link and the corpus would quietly drift un-green, undermining the product's own promise. The dogfood signal was advisory, not a gate. + +## Requirements + +- [REQ-001] Continuous integration runs `proofkeeper coverage` against Proofkeeper's own corpus on every push and pull request, and fails the build when any capability is unverified. +- [REQ-002] The gate uses Proofkeeper's own published CLI behaviour (coverage exits non-zero when a capability has no verifying test), not a bespoke script, so Proofkeeper verifies itself the same way it verifies any consumer. +- [REQ-003] The gate obtains the graph through the supported contract path — the `rac` CLI exporting the corpus — rather than a committed snapshot that could drift. +- [REQ-004] The self-verification status is visible from the README as a badge reflecting the gate's latest result on the default branch. + +## Success Metrics + +- A pull request that adds a capability without a verifying test fails CI on the coverage gate. +- The README badge shows the dogfood gate passing on the default branch when the corpus is fully verified. + +## Risks + +- The gate depends on installing the `rac` engine in CI; an upstream rac-core breakage could fail the job for reasons unrelated to Proofkeeper. Mitigation: install the engine from its repository as a contract dependency, and rely on the graph schema_version guard to distinguish a genuine contract break from noise. +- A flaky external install could make the gate noisy. Mitigation: keep the job minimal and cache where possible; the gate's only product step is the deterministic coverage command. + +## Assumptions + +- The `rac` engine is installable from the rac-core repository and exposes `rac export

--graph` (the published contract). +- `proofkeeper coverage` continues to exit non-zero when any capability is unverified. + +## Related Roadmaps + +- autonomous-qa-enhancements + +## Verified By + +- `.github/workflows/dogfood.yml` diff --git a/lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md b/lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md index eb62063..cdddea8 100644 --- a/lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md +++ b/lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md @@ -66,6 +66,12 @@ rediscover its runtime ID, and emit a compiled test that re-loads the extension so an extension's popup/options and page effects are verified end-to-end, not just ordinary web apps. Serves the faithful-tests outcome on a new surface. +### Self-verification coverage gate + +A required CI job that runs `proofkeeper coverage` against Proofkeeper's own +corpus and fails on any unverified capability, with a README badge — so the +dogfood signal becomes an enforced guarantee that can never drift un-green. + ## Success Measures - A pull request shows exactly one Proofkeeper QA comment regardless of how many