You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Extends the ratchet table from #752. One row goes honestly green; three get honest reasons.
The thing this PR is actually about
The table shipped with one green row and four n/a, and all four said the same thing: "runner is Linux/x86_64; the gate's machine target is Darwin/arm64". That phrasing implies a
macOS runner turns four rows green. It does not. Only one of the four is a runner problem.
Row
What it actually lacked
This PR
signature_valid
macOS + codesign + an installed keg — a runner problem
measured for real
mapped bytes
BrainBar daemon + helper + indexed corpus — a fixture problem
reason sharpened
search p50/p95
same
reason sharpened
idle CPU
same
reason sharpened
A synthetic mini-corpus would measure a different thing, which under this table's own rule owes
it a different method label — so the three are not quietly filled in with a borrowed one.
1. signature_valid, measured
A new signatures job on macos-15 (GitHub's arm64 image) installs the published tap keg and runs scripts/release-verify-signatures.sh against it — the same script AGENTS.md makes mandatory at
release time, not a CI reimplementation free to drift from it. It hands the table a JSON report; invalid > 0 is RED and fails both that job and the table.
What it measures, stated in the table itself: it installs the published formula
(etanhey/layers/brainlayer), so the row covers the release path — formula, published sdist, and
Homebrew's relocation pass. Not this PR's tree. That is the failure it is for: homebrew-layers #37 / cae0b4ad left 20 of 442 files with stale signatures because fix_dynamic_linkage aborted
mid-relocation, and nothing in this repo's CI could have seen it.
2. Cost, and the trigger
A hosted macOS runner bills at ~10× Linux minutes, and this keg builds its venv from source —
the formula forces --no-binary for six Rust/C packages (tokenizers, pydantic-core, safetensors, orjson, rpds-py, cbor2). Per-PR is not defensible.
So a ~15 s ubuntu gate job decides, and the macOS job runs only when:
the PR touches a path that can change what a keg ships or how it is signed — pyproject.toml, MANIFEST.in, setup.py, setup.cfg, scripts/release-*, scripts/brainlayer-version-check.sh, .github/workflows/publish.yml, .github/workflows/ratchet.yml; or
the PR carries the ratchet:signatures label (which is why labeled is now in the
trigger types — otherwise the label would sit there doing nothing until the next push).
table still runs on every PR, so the sticky comment never goes missing.
The honest number I do not have yet: the only measured keg-install datapoint is ~53 s on an M4.
A hosted runner is slower and I will not invent a figure for it. This PR triggers its own macOS
job (it touches ratchet.yml), so the first run measures it — I will post the observed duration
here from that run.
Deliberately not added: a schedule trigger. It would catch Homebrew/upstream drift with no PR
attached, but the ratchet table is PR-scoped — a scheduled run has no comment to refresh. Worth a
follow-up with its own reporting surface, not a bolt-on here.
3. The three sharpened reasons
Before: n/a — runner is Linux/x86_64; the gate's machine target is Darwin/arm64
After: n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
The OS/arch predicate is demoted, not deleted — a served stack on Darwin/x86_64 is still off
target, and there is a test for that. And the tail reasons stop promising w13 as the fix, because
w13 deliberately did not build these three.
!cancelled() and neveralways() — on every writer step and now on every job.
needs: [gate, signatures] orders the jobs; if: ${{ !cancelled() }} is what keeps table
unconditional. A skipped signatures job must not skip table and leave the previous commit's
table standing.
A promised measurement that does not arrive is RED, never n/a.--signature-report asserts
the job ran; a missing, unreadable, or count-less report is a finding.
A gate that never decided is RED too — "not triggered" would be a claim this run never
established.
Real step conditions parsed with pyyaml, no counting assertions.
Baselines stay in Notes with machine + method + date.
One comment per PR, refreshed in place. No second comment, no second table.
Verification — the shipped text, not a mock
Each run: block was extracted from ratchet.yml with pyyaml and executed:
Scenario
Result
real 1.5.11 keg, verify step
442 valid / 0 invalid, verdict=clean → 🟢 GREEN
brew install failed
verdict=unmeasured → 🔴 RED, names the stage
verifier exits 2 without counts
verdict=unmeasured → 🔴 RED, quotes its stderr
gate said no
⚪ n/a carrying the gate's own reason, rc 0
gate did not finish
🔴 RED, rc 1
signatures ran, published nothing
🔴 RED, rc 1
430 valid / 2 invalid
🔴 RED naming cramjam.cpython-313-darwin.so, rc 1
gate path regex
triggers on all 5 release paths, skips docs/ci.yml/ordinary Python, and rejects the decoys subproject/pyproject.toml and src/brainlayer/release_notes.py
92 passed in tests/test_ci_ratchet_table.py (was 50), ruff check and ruff format --check
clean, pre-push gate green.
Two gaps these dry runs caught, both fixed in ecb18016: mapfile is bash ≥ 4 and macOS ships 3.2
(so the Collect step could not be reproduced locally on a Mac), and keying the job's failure off the
invalid count left unmeasured looking like a pass.
Size
M — 226 workflow lines, 262 collector lines, 377 test lines. The test file is the majority and
the workflow is heavily commented because #752 took four review rounds on exactly these conditions.
Reviewers: @cursor review and CodeRabbit. No @codex (2% budget until ~Sun 09-07) and no @bugbot — AGENTS.md forbids routing mandatory reviews there.
Medium Risk
Changes release-path CI and macOS runner spend with security-sensitive codesign gating; mis-gating or a green job on unmeasured signatures would weaken release confidence, though behavior is heavily tested and fail-closed.
Overview Adds real signature_valid measurement by splitting ratchet.yml into three jobs: an Ubuntu gate (release/signing path diff or ratchet:signatures label), an optional macos-15 signatures job that installs the published tap keg and runs the base commit’s release-verify-signatures.sh, and the existing table job that still posts on every PR.
The table job passes --signature-report or --signature-unavailable into ci_ratchet_table.py, which now renders GREEN/RED from parity JSON (invalid counts, empty sweeps, and malformed hand-offs fail closed) and n/a only when the gate opted out. Mapped bytes, search latency, and idle CPUn/a text now leads with the missing BrainBar served stack, not “runner is Linux.”
Workflow hardening moves scratch artifacts to $RUNNER_TEMP, keeps table running when signatures is skipped (needs + if: !cancelled()), verifies even when brew install exits non-zero, and re-triggers on labeled.
Reviewed by Cursor Bugbot for commit 4cbfe53. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add macOS signature_valid measurement to ratchet workflow and clarify unavailable rows
Adds a gate job that inspects changed release/signing paths or the ratchet:signatures label, then decides whether a macOS signature-parity job runs.
Adds a signatures job on macos-15 that installs the published tap formula and runs the base commit's scripts/release-verify-signatures.sh against the installed keg; the job fails on invalid or unmeasured results.
Rewrites row_signature_valid in ci_ratchet_table.py to render measured counts (GREEN only when all counted extensions are valid), RED for failed or malformed reports, and n/a only when the workflow did not request measurement.
Replaces the old runner-side/w13 reasons for row_mapped_bytes, row_search_latency, and row_idle_cpu with a served_stack_requirements prerequisite that names the missing BrainBar served stack, helper, indexed corpus, hosted-runner limitation, and self-hosted installed Mac.
Moves workflow scratch files (reports, tables, API bodies) from checkout-relative paths to $RUNNER_TEMP; intermediate reports and the final PR comment are now built there.
Risk: select_signature fail-closes a promised-but-missing or invalid report as RED instead of falling back to n/a; callers that relied on unconditional n/a for signature_valid will now see findings when a report path is supplied but unreadable.
… other three rows really lack
The ratchet table's four `n/a` rows all read "runner is Linux/x86_64; the gate's machine target is
Darwin/arm64", which made it look like a macOS runner would turn four rows green. It would not.
Only one of the four is a runner problem.
`signature_valid` is now measured for real. A new macOS job installs the published tap keg
(`brew install etanhey/layers/brainlayer`) and runs `scripts/release-verify-signatures.sh` against
it -- the same script AGENTS.md makes mandatory at release time, not a CI reimplementation free to
drift from it. It hands the table a JSON report; `invalid > 0` is RED and fails both that job and
the table.
The other three -- mapped bytes, search p50/p95, idle CPU -- are FIXTURE-bound, not runner-bound.
They need the BrainBar daemon, its hybrid helper and the indexed corpus running together, which no
hosted runner of any OS has. Their reasons now say that, name a self-hosted installed Mac as what
would give them one, and stop promising w13 as the fix -- w13 deliberately did not build them,
because a synthetic corpus measures a different thing and would owe the table a different method
label.
Cost: a hosted macOS runner bills at ~10x Linux minutes and this keg builds its venv from source,
so the job is trigger-gated by a ~15 s ubuntu `gate` job -- release/signing paths, or the
`ratchet:signatures` label. `table` keeps publishing on every PR regardless.
Inherited law from #752 is preserved: `!cancelled()` and never `always()` on every writer and every
job; a promised measurement that does not arrive is RED, never `n/a`; and a gate that never decided
is RED too, because "not triggered" would be a claim this run never established.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-4 dependency
Two gaps the local dry runs surfaced, both against the shipped workflow text rather than a mock:
- `mapfile` is bash >= 4 and macOS ships bash 3.2, so the Collect step could not be reproduced
locally on a Mac. Replaced with an equivalent `read` loop, pinned by a test.
- Keying the signature job's failure off the invalid COUNT let the second failure mode through: an
install that worked but a verifier that exited 2 without counts published a `failed` report and
left the job GREEN. The count is now one of three `verdict` values, defaulting to `unmeasured`,
and only `clean` passes. The table was already RED in that case; the job that discovered it is
no longer green beside it.
Verified by extracting each `run:` block out of the workflow with pyyaml and executing it:
442 valid / 0 invalid against the real 1.5.11 keg (verdict clean), a failed install and a
countless verifier (both verdict unmeasured), and all four hand-off branches feeding the collector
n/a / RED / RED / RED / GREEN with matching exit codes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
We reviewed changes in 0bf0167...4cbfe53 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
The reason will be displayed to describe this comment to others. Learn more.
`select_signature` has a cyclomatic complexity of 20 with "high" risk
A function with high cyclomatic complexity can be hard to understand and
maintain. Cyclomatic complexity is a software metric that measures the number of
independent paths through a function. A higher cyclomatic complexity indicates
that the function has more decision points and is more complex.
The reason will be displayed to describe this comment to others. Learn more.
🟠 Highscripts/ci_ratchet_table.py:547
A PR can modify scripts/release-verify-signatures.sh to emit valid: 1 and invalid: 0, causing this branch to publish GREEN without running codesign. The measured report is trusted solely by its counts, so the check must run a trusted verifier against the published keg and authenticate that provenance before accepting the report.
Also found in 1 other location(s)
.github/workflows/ratchet.yml:150
The verifier is executed from the PR checkout rather than from the installed published keg. A PR that modifies scripts/release-verify-signatures.sh (which also triggers this job) can make it print valid: 1 and invalid: 0, so this step emits a clean report without running codesign; the signature check can therefore be forged by the change it is intended to gate.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @scripts/ci_ratchet_table.py around line 547:
A PR can modify `scripts/release-verify-signatures.sh` to emit `valid: 1` and `invalid: 0`, causing this branch to publish `GREEN` without running `codesign`. The `measured` report is trusted solely by its counts, so the check must run a trusted verifier against the published keg and authenticate that provenance before accepting the report.
Also found in 1 other location(s):
- .github/workflows/ratchet.yml:150 -- The verifier is executed from the PR checkout rather than from the installed published keg. A PR that modifies `scripts/release-verify-signatures.sh` (which also triggers this job) can make it print `valid: 1` and `invalid: 0`, so this step emits a clean report without running `codesign`; the signature check can therefore be forged by the change it is intended to gate.
Every Value below was measured by this run. A row this machine cannot measure says n/a — <reason> instead of a number; baselines in Notes name their own machine, method and date and were not measured here.
Row
Status
Value (measured by this run)
Method
Notes
provenance
🟢 GREEN
stamped 98810e50abc4 == HEAD, tree clean
wheel stamp · in-process · runner
Sha half of #749 keg-mode provenance: a keg built from this wheel can answer __build_sha__. The helper-age and served-process predicates need a running BrainBar and are measured only by scripts/sprint_gate.py on an installed Mac.
mapped bytes
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
Baseline 26.2 GB — installed Mac, socket, 2026-09-03, after R2 drained 15,070 → 0. Up from 16.8 GB because the drain left more vectors mapped under the same cap: the change is the drain, not a leak. Not measured by this run.
search p50/p95
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
Baseline p50 911.887 ms / p95 3167.985 ms, captured 2026-09-01T08:42:22Z on MacBook-Pro.local under active_sprint_load (tests/fixtures/sprint_gate/corpus.json). Not measured by this run.
idle CPU
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
ps sampling · installed Mac
Budget: average CPU < 30% over a 60 s window (resource_budget in scripts/sprint_gate.py). Needs the BrainBar daemon, helper and watcher actually running. Not measured by this run.
signature_valid
🟢 GREEN
442 valid / 0 invalid · brainlayer 1.5.11 · macos-15/ARM64 · brew install exited non-zero (outcome: failure); the keg installed and the sweep ran after it
codesign · brew keg · GitHub macOS runner
scripts/release-verify-signatures.sh <keg> codesign-verifies every *.so/*.dylib under libexec/venv. The macOS parity job installs the published tap formula (etanhey/layers/brainlayer), so this row measures the release path — formula, published sdist and Homebrew's relocation — and not this PR's tree. Release-time baseline for the same keg on a different machine: 442 valid / 0 invalid — installed Mac (M4 Max), brew --prefix brainlayer 1.5.11, 2026-09-03.
🟢 GREEN measured, within budget · 🔴 RED measured, out of budget — a finding to clear before merge · ⚪ n/a not measurable on this machine, never guessed.
No RED rows.
Measured on Linux/x86_64 · checked-out HEAD 98810e50abc4 · run · updated 2026-09-03 16:58:05 UTC
… of the checkout
Both defects were found by this workflow's own first run on #753 — the ratchet caught its author
twice, which is the strongest evidence yet that the table works.
1. `signature_valid` was RED with "the keg did not install". It had. Homebrew `ofail`s a relocation
failure — `MachO::HeaderPadError` on `cramjam.cpython-313-darwin.so`, the exact condition
homebrew-layers #37 documents — so `brew install` exits 1 while STILL installing the keg and
running `post_install`, which is where the codesign sweep lives. Gating the sweep on the install
step's exit code refused to measure the precise post-relocation state this row exists to check.
Measurability now depends on whether there is a `libexec/venv` to sweep, not on brew's exit code.
A clean sweep after an aborted relocation is the #37 fix WORKING, so it stays GREEN — but the
value cell and a job warning both say brew exited non-zero, so a green row cannot hide it.
2. `provenance` was RED with "stamping dirtied the tree". Also mine: the hand-off step wrote
`signature-args.txt` and `signature-report.json` into the checkout, and provenance reads
`git status --porcelain --untracked-files=all`. Every scratch file in all three jobs now goes to
`$RUNNER_TEMP`, pinned by two tests — one listing the known scratch names, one catching a new
one nobody thought to list.
Verified against the real 1.5.11 keg by extracting the shipped `run:` block and executing it:
brew-ofail'd -> 442 valid / 0 invalid, verdict clean; brew clean -> same; no keg -> unmeasured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The measurement is right — `signature_valid` went GREEN at `442 valid / 0 invalid` on the runner,
matching the M4 baseline from a different machine. But the macOS job still reported failure, and
its ONLY failing step was `brew install`: Homebrew `ofail`s the cramjam relocation and exits 1 with
the keg installed and `post_install` swept. Verify succeeded; the RED gate was correctly skipped.
Left alone that job is red on every single run for a condition the formula already handles, and a
permanently red job is a signal reviewers learn to ignore. `continue-on-error: true` on the install
step fixes it without swallowing anything: `steps.install.outcome` is the value BEFORE
continue-on-error, so the verify step still reads `failure`, still emits its warning, and still
records `install_outcome` in the report that the table prints. A missing keg still lands
`verdict=unmeasured`, which fails the job.
Observed cost from run 33777070902: keg install ~8m 45s on macos-15/ARM64 (vs ~53 s on the M4),
gate 4 s, table 15 s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
That independently corroborates the release-time baseline in the row's Notes (442 valid / 0 invalid
on an M4 Max) from a completely different machine. One row honestly green. The other three carry
sharpened reasons naming the daemon + helper + corpus, not Linux.
~9 min × ~10× ≈ 90 Linux-minute-equivalents per triggered run. That confirms the trigger gate
was the right call — and puts a number on it rather than a hand-wave. The gate itself costs 4 s on
every PR.
Three defects this workflow found in itself
The first two were found by the run on 4ddb251d, which came back with 2 RED rows — both mine.
signature_valid RED: "the keg did not install". It had. Homebrew ofails the MachO::HeaderPadError on cramjam.cpython-313-darwin.so — the exact condition
homebrew-layers fix: brain_digest embed method AttributeError #37 documents — so brew install exits 1 with the keg installed and post_install swept. Gating the sweep on the install step's exit code refused to measure the
precise post-relocation state this row exists to check. Fixed in e78be52c: measurability now
depends on whether there is a libexec/venv to sweep.
A clean sweep after an aborted relocation is the fix: brain_digest embed method AttributeError #37 fix working, so it is GREEN — and the
value cell and a job warning both say brew exited non-zero, so the green cannot hide it. Pushing
back on that reading is fair; it is a deliberate call, not an accident.
provenance RED: "stamping dirtied the tree". Also mine — the hand-off step wrote signature-args.txt and signature-report.json into the checkout, and provenance reads git status --porcelain --untracked-files=all. All scratch in all three jobs now goes to $RUNNER_TEMP, pinned by two tests: one listing the known names, one catching a new name nobody
thought to list.
The macOS job was red while its measurement was correct. Its only failing step was brew install. Left alone, this job is red on every run for a condition the formula already
handles, and a permanently red job is a signal reviewers learn to ignore. 6ea5b395 adds continue-on-error: true to that step — steps.install.outcome is the value before continue-on-error, so verify still reads failure, still warns, and still records install_outcome. A missing keg still lands verdict=unmeasured, which fails the job.
One thing to flag, and it is NOT in this PR
provenance's dirty-tree message hardcodes "brainlayer/_build.py must stay gitignored" for any
dirty tree. It misdiagnosed defect 2 above — the dirty file was signature-args.txt, and the message
sent me looking at .gitignore. That message is #752 code, and repo law says a defect found in other
code opens a new PR, so it is deliberately untouched here. Worth a one-line fix that names the
actually-dirty paths.
Verification
98 passed in tests/test_ci_ratchet_table.py (was 50). Every run: block was extracted from ratchet.yml with pyyaml and executed against the real 1.5.11 keg — brew-ofail'd, brew-clean,
and no-keg-at-all — plus all four hand-off branches and the gate's path regex against 10 file lists
including two decoys. ruff check and ruff format --check clean; pre-push gate green.
Macroscope found four and DeepSource one. All five were real; none are style.
HIGH — the verifier was forgeable by the PR it gates. `scripts/release-*` is one of the paths that
TRIGGERS this job, and the script ran from the PR checkout, so a PR could rewrite it to print
`valid: 1 / invalid: 0` and take signature_valid GREEN with codesign never invoked. The signatures
job now checks out `github.event.pull_request.base.sha` at `trusted/` and runs the verifier from
there. Trade-off stated in the code and in the step summary on every run: a PR that legitimately
improves the verifier is measured by the trusted copy, so its own change is reviewed by humans and
the suite rather than by this row.
HIGH — `gh api repos/.../pulls/N/files` caps at 3,000 files however you paginate, so a release path
past the cap made the gate answer `false` and skip the macOS job on exactly the large PR that needed
it — while reporting a confident "touches no release path" reason for a list it never fully saw. The
gate now diffs locally (`git diff --name-only "$BASE_SHA" HEAD`, checkout at fetch-depth 0), which
has no cap, and fails closed if the base commit is absent.
MEDIUM — release-verify-signatures.sh PRINTS `valid: 0 / invalid: 0` and only THEN exits 1 on an
empty sweep, so the job called that `clean` while the collector rendered the row RED. Two CI
statuses contradicting each other about one measurement. 0/0 is now `unmeasured` in both.
MEDIUM — a count past sys.get_int_max_str_digits() raises a bare ValueError out of json.loads (not
a JSONDecodeError), and `invalid_files: 1` raised TypeError out of the tuple comprehension. Either
aborted the whole table, including the provenance row this job did measure. Both are now fail-closed
RED, with a test that walks seven malformed hand-off shapes and asserts a table still renders.
DeepSource — select_signature's cyclomatic complexity was 20. Split into read_signature_payload /
signature_failure / signature_measurement, and row_signature_valid's rendering into signature_value
/ measured_signature_row. Max complexity in the file is now select_wheel's pre-existing 13.
Re-verified against the real 1.5.11 keg through the trusted path (442 valid / 0 invalid,
verdict clean) and with a venv containing no native extensions (verdict unmeasured, collector RED).
106 tests pass, up from 98.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
Overlapping exceptions (ValueError is an ancestor class of UnicodeDecodeError)
Two or more exceptions in the same exception handler clause that are identical or parts of the same inheritance hierarchy (i.e. overlapping). It is recommmend to refactor this.
The reason will be displayed to describe this comment to others. Learn more.
Unnecessary use of a comprehension, use dict(_signature_fields(report)) instead.
It is unnecessary to use a comprehension just to loop over the iterable and create a list/set/dict out of it. Python has a specialized set of tools for this task: the list/set/dict constructors, which are faster and more readable.
Two DeepSource nits on the previous commit, both correct.
`UnicodeDecodeError` subclasses `ValueError`, so listing both was redundant. Narrowed to
`(OSError, ValueError)` with a comment naming all three failures ValueError actually covers here —
JSONDecodeError, UnicodeDecodeError, and the int-digit-limit ValueError. Confirmed a non-UTF-8
report still renders RED with `(UnicodeDecodeError)` named in the value.
The complexity finding DeepSource still shows is stale — it is anchored to 6ea5b39, before the
split; the tool re-analyzed 01fd9d3 and did not re-raise it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![deepsource-io[bot]](https://avatars.githubusercontent.com/in/16372?s=60&v=4){kind=small}
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
Extends the ratchet table from #752. One row goes honestly green; three get honest reasons.
The thing this PR is actually about
The table shipped with one green row and four
n/a, and all four said the same thing:"runner is Linux/x86_64; the gate's machine target is Darwin/arm64". That phrasing implies a
macOS runner turns four rows green. It does not. Only one of the four is a runner problem.
signature_validcodesign+ an installed keg — a runner problemmapped bytessearch p50/p95idle CPUA synthetic mini-corpus would measure a different thing, which under this table's own rule owes
it a different method label — so the three are not quietly filled in with a borrowed one.
1.
signature_valid, measuredA new
signaturesjob onmacos-15(GitHub's arm64 image) installs the published tap keg and runsscripts/release-verify-signatures.shagainst it — the same scriptAGENTS.mdmakes mandatory atrelease time, not a CI reimplementation free to drift from it. It hands the table a JSON report;
invalid > 0is RED and fails both that job and the table.What it measures, stated in the table itself: it installs the published formula
(
etanhey/layers/brainlayer), so the row covers the release path — formula, published sdist, andHomebrew's relocation pass. Not this PR's tree. That is the failure it is for: homebrew-layers
#37 /
cae0b4adleft 20 of 442 files with stale signatures becausefix_dynamic_linkageabortedmid-relocation, and nothing in this repo's CI could have seen it.
2. Cost, and the trigger
A hosted macOS runner bills at ~10× Linux minutes, and this keg builds its venv from source —
the formula forces
--no-binaryfor six Rust/C packages (tokenizers,pydantic-core,safetensors,orjson,rpds-py,cbor2). Per-PR is not defensible.So a ~15 s ubuntu
gatejob decides, and the macOS job runs only when:pyproject.toml,MANIFEST.in,setup.py,setup.cfg,scripts/release-*,scripts/brainlayer-version-check.sh,.github/workflows/publish.yml,.github/workflows/ratchet.yml; orratchet:signatureslabel (which is whylabeledis now in thetrigger
types— otherwise the label would sit there doing nothing until the next push).tablestill runs on every PR, so the sticky comment never goes missing.The honest number I do not have yet: the only measured keg-install datapoint is ~53 s on an M4.
A hosted runner is slower and I will not invent a figure for it. This PR triggers its own macOS
job (it touches
ratchet.yml), so the first run measures it — I will post the observed durationhere from that run.
Deliberately not added: a
scheduletrigger. It would catch Homebrew/upstream drift with no PRattached, but the ratchet table is PR-scoped — a scheduled run has no comment to refresh. Worth a
follow-up with its own reporting surface, not a bolt-on here.
3. The three sharpened reasons
Before:
n/a — runner is Linux/x86_64; the gate's machine target is Darwin/arm64After:
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac wouldThe OS/arch predicate is demoted, not deleted — a served stack on Darwin/x86_64 is still off
target, and there is a test for that. And the tail reasons stop promising w13 as the fix, because
w13 deliberately did not build these three.
Inherited law from #752, preserved
!cancelled()and neveralways()— on every writer step and now on every job.needs: [gate, signatures]orders the jobs;if: ${{ !cancelled() }}is what keepstableunconditional. A skipped
signaturesjob must not skiptableand leave the previous commit'stable standing.
n/a.--signature-reportassertsthe job ran; a missing, unreadable, or count-less report is a finding.
established.
Verification — the shipped text, not a mock
Each
run:block was extracted fromratchet.ymlwith pyyaml and executed:verifystep442 valid / 0 invalid,verdict=clean→ 🟢 GREENbrew installfailedverdict=unmeasured→ 🔴 RED, names the stageverdict=unmeasured→ 🔴 RED, quotes its stderrn/acarrying the gate's own reason, rc 0430 valid / 2 invalidcramjam.cpython-313-darwin.so, rc 1ci.yml/ordinary Python, and rejects the decoyssubproject/pyproject.tomlandsrc/brainlayer/release_notes.py92 passedintests/test_ci_ratchet_table.py(was 50),ruff checkandruff format --checkclean, pre-push gate green.
Two gaps these dry runs caught, both fixed in
ecb18016:mapfileis bash ≥ 4 and macOS ships 3.2(so the Collect step could not be reproduced locally on a Mac), and keying the job's failure off the
invalid count left
unmeasuredlooking like a pass.Size
M — 226 workflow lines, 262 collector lines, 377 test lines. The test file is the majority and
the workflow is heavily commented because #752 took four review rounds on exactly these conditions.
Reviewers:
@cursor reviewand CodeRabbit. No@codex(2% budget until ~Sun 09-07) and no@bugbot—AGENTS.mdforbids routing mandatory reviews there.🤖 Generated with Claude Code
Note
Medium Risk
Changes release-path CI and macOS runner spend with security-sensitive codesign gating; mis-gating or a green job on unmeasured signatures would weaken release confidence, though behavior is heavily tested and fail-closed.
Overview
Adds real
signature_validmeasurement by splittingratchet.ymlinto three jobs: an Ubuntu gate (release/signing path diff orratchet:signatureslabel), an optionalmacos-15signatures job that installs the published tap keg and runs the base commit’srelease-verify-signatures.sh, and the existing table job that still posts on every PR.The table job passes
--signature-reportor--signature-unavailableintoci_ratchet_table.py, which now renders GREEN/RED from parity JSON (invalid counts, empty sweeps, and malformed hand-offs fail closed) and n/a only when the gate opted out. Mapped bytes, search latency, and idle CPUn/atext now leads with the missing BrainBar served stack, not “runner is Linux.”Workflow hardening moves scratch artifacts to
$RUNNER_TEMP, keepstablerunning whensignaturesis skipped (needs+if: !cancelled()), verifies even whenbrew installexits non-zero, and re-triggers onlabeled.Reviewed by Cursor Bugbot for commit 4cbfe53. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add macOS
signature_validmeasurement to ratchet workflow and clarify unavailable rowsgatejob that inspects changed release/signing paths or theratchet:signatureslabel, then decides whether a macOS signature-parity job runs.signaturesjob onmacos-15that installs the published tap formula and runs the base commit'sscripts/release-verify-signatures.shagainst the installed keg; the job fails on invalid or unmeasured results.row_signature_validin ci_ratchet_table.py to render measured counts (GREEN only when all counted extensions are valid), RED for failed or malformed reports, andn/aonly when the workflow did not request measurement.row_mapped_bytes,row_search_latency, androw_idle_cpuwith aserved_stack_requirementsprerequisite that names the missing BrainBar served stack, helper, indexed corpus, hosted-runner limitation, and self-hosted installed Mac.$RUNNER_TEMP; intermediate reports and the final PR comment are now built there.select_signaturefail-closes a promised-but-missing or invalid report as RED instead of falling back ton/a; callers that relied on unconditionaln/aforsignature_validwill now see findings when a report path is supplied but unreadable.Macroscope summarized 4cbfe53.