Skip to content

Commit 14bae0d

Browse files
authored
Merge pull request #24 from itsthelore/claude/dogfood-gate
chore(release): enforce self-coverage in CI + dogfooding badge
2 parents 18c7a05 + b1ce759 commit 14bae0d

5 files changed

Lines changed: 149 additions & 0 deletions

File tree

.github/workflows/dogfood.yml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Dogfooding
2+
3+
# Proofkeeper verifies itself: run its own `coverage` command against its own
4+
# Lore corpus (lore-proofkeeper/) and fail if any capability is unverified. This
5+
# turns the dogfood signal into an enforced gate — the corpus can never drift
6+
# un-green. It exercises the real CLI and the real `rac export --graph` contract
7+
# path (ADR-063, ADR-083), not a bespoke checker.
8+
9+
on:
10+
push:
11+
branches: ["**"]
12+
pull_request:
13+
14+
jobs:
15+
coverage:
16+
name: self-coverage (proofkeeper verifies proofkeeper)
17+
runs-on: ubuntu-latest
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- uses: actions/setup-node@v4
22+
with:
23+
node-version: "20"
24+
cache: npm
25+
- run: npm ci
26+
- run: npm run build
27+
28+
# The `rac` engine is a contract dependency, not a package dependency:
29+
# install it as an external CLI from rac-core (its distribution is
30+
# "requirements-as-code"; the console script is `rac`). Proofkeeper consumes
31+
# only its published `rac export --graph` JSON output.
32+
- uses: actions/setup-python@v5
33+
with:
34+
python-version: "3.11"
35+
- name: Install the rac engine (provides the `rac` CLI)
36+
run: pip install "git+https://github.com/itsthelore/rac-core.git"
37+
38+
- name: Proofkeeper coverage over its own corpus (fails on any unverified)
39+
run: node dist/cli.js coverage --corpus lore-proofkeeper/

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Banner: add docs/assets/proofkeeper-header-{dark,light}.png, then uncomment.
2020

2121
<p align="center">
2222
<a href="https://github.com/itsthelore/proofkeeper/actions/workflows/ci.yml"><img src="https://github.com/itsthelore/proofkeeper/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
23+
<a href="https://github.com/itsthelore/proofkeeper/actions/workflows/dogfood.yml"><img src="https://img.shields.io/github/actions/workflow/status/itsthelore/proofkeeper/dogfood.yml?branch=main&label=dogfooding&logo=githubactions&logoColor=white" alt="Dogfooding: Proofkeeper verifies its own corpus"></a>
2324
<a href="https://www.npmjs.com/package/@itsthelore/proofkeeper"><img src="https://img.shields.io/npm/v/@itsthelore/proofkeeper" alt="npm"></a>
2425
<img src="https://img.shields.io/badge/node-%E2%89%A520-blue" alt="Node >= 20">
2526
<img src="https://img.shields.io/badge/types-TypeScript-blue.svg" alt="TypeScript">
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
schema_version: 1
3+
id: PK-KWBH4HMAVW38
4+
type: design
5+
---
6+
# Self-Verification Coverage Gate
7+
8+
## Context
9+
10+
Proofkeeper records its own capabilities in `lore-proofkeeper/` and proves them with `## Verified By` links, but the dogfood report was never enforced. This design makes "Proofkeeper's corpus stays fully verified" a required CI gate, using Proofkeeper's own coverage command, and surfaces it as a README badge.
11+
12+
## User Need
13+
14+
A maintainer (and a reader evaluating the project) needs confidence that Proofkeeper actually practises what it sells — that every capability it claims is backed by a test — without manually running coverage and without trusting an advisory number that can silently drift.
15+
16+
## Design
17+
18+
- A dedicated workflow `.github/workflows/dogfood.yml`, named so its status badge reads as the dogfood signal. On push and pull request it:
19+
- checks out the repo, sets up Node, `npm ci`, `npm run build`;
20+
- sets up Python and installs the `rac` engine from the rac-core repository (`pip install "git+https://github.com/itsthelore/rac-core.git"`), which provides the `rac` command;
21+
- runs `node dist/cli.js coverage --corpus lore-proofkeeper/`. The command exits non-zero if any capability is unverified, so the job fails on drift with no extra scripting.
22+
- The README adds a badge pointing at this workflow's status on the default branch (a shields.io GitHub-Actions badge with a `dogfooding` label), next to the existing CI and npm badges.
23+
- This requirement is itself verified by the workflow file, keeping the corpus self-consistent (every capability, including this one, has a `## Verified By`).
24+
25+
## Constraints
26+
27+
- Reuse the published behaviour: the gate runs the real `coverage` CLI and the real `rac export` contract path, not a bespoke checker (ADR-063, ADR-083).
28+
- No engine import: `rac` is installed as an external CLI; Proofkeeper consumes its JSON output only.
29+
- Determinism over snapshots: export the corpus live rather than committing a graph file that could drift from the artifacts.
30+
31+
## Rationale
32+
33+
Enforcing the existing coverage command is the smallest change that turns an advisory signal into a guarantee, and it dogfoods the product end to end — Proofkeeper verifies itself exactly as a consumer would. A workflow-status badge is honest because it reflects a real run, not a hand-set value.
34+
35+
## Alternatives
36+
37+
- **Commit a `rac export --graph` snapshot and check against it.** Rejected: the snapshot drifts from the artifacts and would need its own freshness gate.
38+
- **A custom script that parses the corpus directly.** Rejected: it would bypass the very CLI and contract path the product ships, weakening the dogfood.
39+
- **A manually maintained "passing" badge.** Rejected: not tied to a real result; can lie.
40+
41+
## Accessibility
42+
43+
The badge carries descriptive alt text; status is conveyed by text ("passing"/"failing"), not colour alone.
44+
45+
## Style Guidance
46+
47+
Match the existing workflow conventions in `.github/workflows/ci.yml` (Node 20, `npm ci`, named jobs) and place the badge alongside the current badge row.
48+
49+
## Open Questions
50+
51+
- Whether to pin the rac-core install to a released tag for extra determinism once a stable cadence exists. Deferred; track via the schema_version guard for now.
52+
53+
## Related Requirements
54+
55+
- req-dogfood-coverage-gate
56+
57+
## Related Roadmaps
58+
59+
- autonomous-qa-enhancements
60+
61+
## Status
62+
63+
Accepted
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
schema_version: 1
3+
id: PK-KWBH4H0WB6VH
4+
type: requirement
5+
---
6+
# Self-Verification Coverage Gate
7+
8+
## Problem
9+
10+
Proofkeeper dogfoods itself: its own capabilities live as requirements in `lore-proofkeeper/`, and `proofkeeper coverage --corpus lore-proofkeeper/` reports which have a verifying test. But nothing enforced that report — a capability could be added without a `## Verified By` link and the corpus would quietly drift un-green, undermining the product's own promise. The dogfood signal was advisory, not a gate.
11+
12+
## Requirements
13+
14+
- [REQ-001] Continuous integration runs `proofkeeper coverage` against Proofkeeper's own corpus on every push and pull request, and fails the build when any capability is unverified.
15+
- [REQ-002] The gate uses Proofkeeper's own published CLI behaviour (coverage exits non-zero when a capability has no verifying test), not a bespoke script, so Proofkeeper verifies itself the same way it verifies any consumer.
16+
- [REQ-003] The gate obtains the graph through the supported contract path — the `rac` CLI exporting the corpus — rather than a committed snapshot that could drift.
17+
- [REQ-004] The self-verification status is visible from the README as a badge reflecting the gate's latest result on the default branch.
18+
19+
## Success Metrics
20+
21+
- A pull request that adds a capability without a verifying test fails CI on the coverage gate.
22+
- The README badge shows the dogfood gate passing on the default branch when the corpus is fully verified.
23+
24+
## Risks
25+
26+
- The gate depends on installing the `rac` engine in CI; an upstream rac-core breakage could fail the job for reasons unrelated to Proofkeeper. Mitigation: install the engine from its repository as a contract dependency, and rely on the graph schema_version guard to distinguish a genuine contract break from noise.
27+
- A flaky external install could make the gate noisy. Mitigation: keep the job minimal and cache where possible; the gate's only product step is the deterministic coverage command.
28+
29+
## Assumptions
30+
31+
- The `rac` engine is installable from the rac-core repository and exposes `rac export <dir> --graph` (the published contract).
32+
- `proofkeeper coverage` continues to exit non-zero when any capability is unverified.
33+
34+
## Related Roadmaps
35+
36+
- autonomous-qa-enhancements
37+
38+
## Verified By
39+
40+
- `.github/workflows/dogfood.yml`

lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,12 @@ rediscover its runtime ID, and emit a compiled test that re-loads the extension
6666
so an extension's popup/options and page effects are verified end-to-end, not just
6767
ordinary web apps. Serves the faithful-tests outcome on a new surface.
6868

69+
### Self-verification coverage gate
70+
71+
A required CI job that runs `proofkeeper coverage` against Proofkeeper's own
72+
corpus and fails on any unverified capability, with a README badge — so the
73+
dogfood signal becomes an enforced guarantee that can never drift un-green.
74+
6975
## Success Measures
7076

7177
- A pull request shows exactly one Proofkeeper QA comment regardless of how many

0 commit comments

Comments
 (0)