Skip to content

fix: enforce sprint gate machine target - #745

Merged
EtanHey merged 2 commits into
mainfrom
feat/gate-machine-target
Sep 1, 2026
Merged

fix: enforce sprint gate machine target#745
EtanHey merged 2 commits into
mainfrom
feat/gate-machine-target

Conversation

@EtanHey

@EtanHey EtanHey commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • Enforce the corpus machine_target before a live gate run touches any check.
  • Choose option (b): keep the single MacBook-Pro latency baseline and report search_latency as SKIPPED on any other calibrated-target host while MCP, resource, and WAL checks still run.
  • Fail with machine-readable JSON when required machine or latency-baseline configuration is missing, incomplete, or not an object, and keep deterministic --fixture replays exempt from live-host enforcement.

Portability decision

Option (b) is the smallest honest shape because no second-machine latency baseline exists yet. Its consequence is explicit: the M1's latency is never gated even though releases require both Macs. A skipped latency check deliberately leaves status=PASS and rc=0 so the other three standalone checks complete, while the top-level skipped array exposes partial coverage.

The first release or CI consumer that wires this gate up MUST assert skipped == []; otherwise the field is decoration. Option (a), per-hostname latency baselines for both Macs, is the tracked follow-up.

Known rowed defects

  • checks: [] currently produces a vacuous PASS.

This remains deliberately rowed to PR-0e under the tight-loop rule.

Test plan

  • python3 -m pytest -q tests/test_sprint_gate.py — 23 passed.
  • ruff check scripts/sprint_gate.py tests/test_sprint_gate.py — clean.
  • ruff format --check scripts/sprint_gate.py tests/test_sprint_gate.py — clean.
  • Changed-only pre-push gate — 23 sprint-gate, 3 MCP registration, 40 isolated routing/eval, 1 Bun, and 1 shell regression passed on bbaddc3f; 0 failures.
  • Mutation proof — unconditional live latency skipping fails test_live_gate_runs_latency_on_calibrated_host; clean branch passes.

Review

  • Lead-routed Claude pair review: two bounded rounds completed; lead directly verified the post-round-2 delta and opened the commit/PR gate.
  • Local CodeRabbit pre-commit reviews: initial 2 minor findings fixed; reversal delta reviewed with 0 findings.
  • Codex and Macroscope bot-round-1 findings: incomplete and non-object configuration validation fixed on bbaddc3f.

— brainlayerCodex-b88e45fa (worker) · codex/gpt-5.6-sol


Note

Medium Risk
Changes release-gate semantics (PASS with skipped latency) and pre-check validation; consumers must enforce skipped == [] to avoid silent partial coverage.

Overview
Live sprint gate runs now validate corpus machine_target and latency_baseline_ms before any check executes, emitting structured JSON failures (empty checks, specific error) when config is missing, invalid, or mismatched with the host. --fixture replays stay exempt.

On hosts that match the machine target but not the baseline’s calibrated hostname, search_latency is reported as SKIPPED (with running vs calibrated hostname in details) while MCP, resource, and WAL checks still run. Overall status can be PASS when checks are only PASS or SKIPPED, and a top-level skipped list names partial coverage—release wiring must treat non-empty skipped as incomplete gating.

Tests add deterministic_live_config / run_live_config helpers and cover mismatch, invalid targets, baseline validation, skip vs run on calibrated host, and baseline optional when latency isn’t selected.

Reviewed by Cursor Bugbot for commit bbaddc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Enforce machine_target validation and skip search_latency on uncalibrated hosts in sprint_gate

  • In live mode, main now validates that machine_target exists, is a dict with at least os and architecture, and exactly matches the running machine (extra keys cause a mismatch)
  • When search_latency is selected, validates latency_baseline_ms is a dict with a hostname; skips the check with a SKIPPED result if the current host doesn't match the calibrated hostname
  • SKIPPED results count as passing for overall status; a skipped list is added to the JSON output payload
  • Validation errors now emit a standardized JSON error payload via a local fail() helper and exit before running checks
  • Fixture (replay) mode bypasses all live validations
  • Risk: live runs on machines not matching machine_target (in sprint_gate.py) now fail with rc=1 instead of proceeding; any config with extra keys in machine_target will fail

Macroscope summarized bbaddc3.

Co-Authored-By: brainlayerCodex-b88e45fa running gpt-5.6-sol <noreply@anthropic.com>
@EtanHey EtanHey added the XS Extra-small change (400 lines or fewer) label Sep 1, 2026

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@cursor

cursor Bot commented Sep 1, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_98948f98-87c5-485b-9c11-4b91d01625e7)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T15:44:46.262501Z bbaddc3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@EtanHey

EtanHey commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

@codex review

— brainlayerCodex-b88e45fa (worker) · codex/gpt-5.6-sol

@deepsource-io

deepsource-io Bot commented Sep 1, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 4fee4b3...bbaddc3 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

Important

Some issues found as part of this review are outside of the diff in this pull request and aren't shown in the inline review comments due to GitHub's API limitations. You can see those issues on the DeepSource dashboard.

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
Python Sep 1, 2026 3:41p.m. Review ↗
Swift Sep 1, 2026 3:41p.m. Review ↗
JavaScript Sep 1, 2026 3:41p.m. Review ↗
Shell Sep 1, 2026 3:41p.m. Review ↗
Secrets Sep 1, 2026 3:41p.m. Review ↗

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

Comment thread scripts/sprint_gate.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ed9c155d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/sprint_gate.py
machine_target = config.get("machine_target")
if not args.fixture and machine_target is None:
return fail("machine target is missing")
if not args.fixture and any(key not in machine or machine[key] != value for key, value in machine_target.items()):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require a complete machine target

When machine_target is empty or omits either os or architecture, this expression validates only the supplied fields (any(...) is false for {}), so the live gate can execute and report PASS on an unintended host. Since this guard is meant to prevent cross-machine results, reject incomplete targets before running any checks.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bbaddc3f. Live mode now rejects an object missing either required os or architecture key before any check runs. Regression coverage includes both {} and {"os":"Darwin"} and asserts rc=1, JSON FAIL, and an empty checks array.

— brainlayerCodex-b88e45fa (worker) · codex/gpt-5.6-sol

Co-Authored-By: brainlayerCodex-b88e45fa running gpt-5.6-sol <noreply@anthropic.com>
@EtanHey

EtanHey commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

@codex review

— brainlayerCodex-b88e45fa (worker) · codex/gpt-5.6-sol

@cursor

cursor Bot commented Sep 1, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_656b609c-ebab-400b-9cbc-f40a0f2d4934)

@EtanHey

EtanHey commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

@codex review

— brainlayerCodex-b88e45fa (worker) · codex/gpt-5.6-sol

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: bbaddc3fc7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 4d4967a0-de4a-4509-9eac-bdf8d3fb26bf


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@EtanHey
EtanHey merged commit 73ea042 into main Sep 1, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

XS Extra-small change (400 lines or fewer)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant