Skip to content

test(#1784): assert the closed-PR review skip stops the stage - #1785

Merged
waynesun09 merged 1 commit into
mainfrom
agent/1784-prove-review-skip
Oct 24, 2026
Merged

waynesun09 merged 1 commit into
mainfrom
agent/1784-prove-review-skip

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Changes in this repository:

  • behaviour/features/review/post-review-outcomes.feature: in "A rejecting review closes the pull request", the second /fs-review round now adds And the harness "review" run was skipped by its pre-script right after the workflow step.
  • go.mod / go.sum: bump github.com/fullsend-ai/fullsend from 6d925afef to fullsend main 3ea9dd1b8 (v0.50.1-0.20261024212938-3ea9dd1b82e4). That version adds the step. The old pin's go.sum entries are replaced, matching previous bumps. fullsend's go.mod hash is unchanged.

Why

Before this change, the second round checked only that the review workflow succeeded and that the "Review skipped" comment was posted. Both pass even if the agent ran anyway. The new step downloads the fullsend-review artifact for that run. It requires fullsend.prescript.skipped=true on the root run span in the host telemetry, and it requires that there is no agent-result.json or behaviour-results.json. So it proves pre-review stopped the stage before the sandbox was created.

Context

Testing

  • The behaviour package (-tags behaviour) vets and compiles against the new pin, using the same -modfile + local replace setup as behaviour-tests.yml.
  • The new feature line matches the registered step regex ^the harness "([^"]+)" run was skipped by its pre-script$ and captures review.
  • harness/triggertest passes against the new pin. It reads the root go.mod pin's fixtures.
  • make test could not run through make (the sandbox blocks /bin/sh). I ran every script-test command directly: 31 pass and the same 15 environment-only failures appear on main and on this branch.
  • skillsaw --strict (the make lint command) passed.
  • pre-commit could not fetch hook repos. I ran the matching pre-commit-hooks v6.0.0 checks and lint-release-cases directly and they passed. scan-secrets (gitleaks) passed.
  • Not run: live behaviour scenarios. The merge criterion (Agents behaviour green with the reject scenario passing, not skipped, plus the job duration) has to be checked in CI.

🤖 Generated with Claude Code


Closes #1784

Post-script verification

  • Branch is not main/master (agent/1784-prove-review-skip)
  • Secret scan passed (gitleaks — 5543e099221f08daf2d1efdbe7cc62efa2c9252b..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

The reject scenario's second /fs-review round only checked that the
review workflow succeeded and posted the "Review skipped" comment.
Both hold whether or not the agent ran, so the scenario could not
prove that pre-review stopped the stage (fixed in #1783).

Bump github.com/fullsend-ai/fullsend to main 3ea9dd1b8
(fullsend-ai/fullsend#8793, for fullsend-ai/fullsend#8792), which
adds the step "the harness <agent> run was skipped by its
pre-script". It reads the run's host telemetry for
fullsend.prescript.skipped=true and requires no agent output. Add it
after the second round's workflow step. The old pin's go.sum entries
are replaced, matching previous bumps; fullsend's go.mod hash is
unchanged.

Verification: the behaviour package vets and compiles against the new
pin with the CI-style -modfile replace setup, and the new line matches
the registered step regex. harness/triggertest passes. Live behaviour
scenarios could not run in the sandbox. make cannot run here (sandbox
blocks /bin/sh); running the script-test commands directly gives the
same 15 environment-only failures on main and on this branch.
skillsaw --strict passed. pre-commit could not fetch hook repos;
pre-commit-hooks v6.0.0 checks and lint-release-cases run directly
passed, and scan-secrets (gitleaks) passed.

Closes #1784

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner October 24, 2026 22:12
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Oct 24, 2026
@waynesun09

Copy link
Copy Markdown
Member

/fs-fix-stop

@github-actions github-actions Bot added the fullsend-no-fix Skip bot-triggered fix agent runs label Oct 24, 2026
@github-actions

Copy link
Copy Markdown

Fix agent disabled for this PR. Remove the fullsend-no-fix label or use /fs-fix to re-engage.

@fullsend-ai-review

fullsend-ai-review Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:14 PM UTC · Completed 10:21 PM UTC

Commit: 7171358 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $0.97

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Oct 24, 2026
@fullsend-ai-review

Copy link
Copy Markdown

<!-- fullsend:risk-assessment -->
Risk Assessment: moderate (2/5)

Details

Small, tightly scoped 3-file PR aligning with issue requirements, with risk elevated to moderate due to dependency updates in go.mod and go.sum alongside recent churn on those manifests.

@fullsend-ai-review

Copy link
Copy Markdown

Looks good to me

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Oct 24, 2026

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 7171358: fullsend bump to include #8793; behaviour run 40137590399 passed 11/11 with the reject scenario now proving the closed-PR review skip stops the stage; bot approved, no threads.

@waynesun09
waynesun09 added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit 15e2d79 Oct 24, 2026
78 of 86 checks passed
@waynesun09
waynesun09 deleted the agent/1784-prove-review-skip branch October 24, 2026 22:34

This branch was successfully deployed

1 active deployment
dev — 7171358c Deployed Oct 24, 2026 by fullsend-ai-coder[bot] via behaviour #1275
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fullsend-no-fix Skip bot-triggered fix agent runs ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(behaviour): prove the closed-PR review skip stops the stage

1 participant