Skip to content

Issue #1152: fix: gate issue-sweep commit evidence on merge-base with main - #1153

Open
bjagg wants to merge 1 commit into
LIF-Initiative:mainfrom
bjagg:fix/issue-sweep-merge-base
Open

Issue #1152: fix: gate issue-sweep commit evidence on merge-base with main#1153
bjagg wants to merge 1 commit into
LIF-Initiative:mainfrom
bjagg:fix/issue-sweep-merge-base

Conversation

@bjagg

@bjagg bjagg commented Aug 19, 2026

Copy link
Copy Markdown
Contributor
Description of Change

Problem. The issue-sweep skill accepted a commit SHA as proof that an open issue was
already resolved, without checking the commit is on main. Its judge prompt directs agents to
git log --oneline --grep "#NNN" / git log -S "<symbol>" for commit evidence, and its stated
evidence bar — "a merged PR #, a commit SHA, or a named shipped file/symbol" — is satisfied by a
SHA from an unmerged branch.

A bare git log --grep resolves from HEAD, not main, so the gap opens two ways: the sweep
is run from a feature branch (HEAD carries that branch's unmerged commits), or the scoped search
comes back empty and the agent widens it with --all, which is a natural move given the prompt
asks it to hunt for evidence. Because the sweep's output is a close-list, the failure mode is
proposing closure of an issue whose work is still on a branch.

Solution — two changes, both required:

  1. Scope the searches. git log main --oneline --grep "#N" and git log main -S "<symbol>".
    This fixes the root cause independent of which branch the sweep runs from, and widening an
    empty search with --all is now explicitly forbidden ("an empty result from main is the
    answer").
  2. Gate the citation. Any SHA must pass git merge-base --is-ancestor <sha> main before it
    can be cited, wherever it was found — a PR body, an issue comment, a wider search.

A failed ancestor check returns open/uncertain with the branch named ("commit <sha> exists
on <branch>, not merged to main") rather than dropping the finding — in-flight work is useful
information, it just isn't a closure. The arbiter previously said only "investigate the same
way" and inherited the gap, so it now re-runs the check on any commit either judge cited. The
#722 case below is inline in the prompt as the worked example, and the rule is restated in the
skill's Rules section.

Side effects / limitations. Documentation-only change to a .claude/ skill file; no runtime
code, no dependency, no schema. The gh pr list --state merged --search evidence path was
already sound and is unchanged. This does not address two other known gaps in the same skill —
it runs two judges where overlap between two agents can't be measured, and its verdict enum has
no PARTIAL — both left for separate PRs.

How reviewers can test this. The bug is reproducible from a clean checkout, against a
currently-open issue:

$ gh issue view 722 --json number,state,title
{"number":722,"state":"OPEN","title":"Investigate and Improve Translator Performance"}

$ git log --all --oneline --grep "#722"
65f48b3 Issue #722: fix: Add cachetools dependency to translator project pyproject.toml
89fd4b0 Issue #722: feat: Add MDR caching, optimize deepcopy, and add performance benchmarks

$ git merge-base --is-ancestor 89fd4b0 main && echo "on main" || echo "NOT on main"
NOT on main

$ git branch -a --contains 89fd4b0
  remotes/upstream/issue-722---Translator-performance

Under the old prompt those SHAs are citable evidence that #722 is resolved. Under the new one the
ancestor check fails and the verdict stays open with the branch named. Repo-wide,
git log --all --oneline --not main | wc -l reports 190 such commits, and the Issue #NNN:
commit convention means each matches a grep for its own issue number.

Related Issues

Closes #1152

Type of Change
  • Bug fix (non-breaking change which fixes an issue)
Project Area(s) Affected
  • Documentation (docs/, READMEs, ARCHITECTURE.md, CLAUDE.md)

Checklist
  • commit message follows commit guidelines (see commitlint.config.mjs)
  • code passes linting checks (uv run ruff check) — skipped by pre-commit, no Python files
  • code passes formatting checks (uv run ruff format) — skipped by pre-commit, no Python files
  • code passes type checking (uv run ty check)
  • pre-commit hooks have been run successfully
Testing
  • Manual testing performed

The changed text lives inside a JS template literal in a fenced code block, so escaping was the
real risk. Verification:

  • Extracted the Workflow script from the skill and ran node --check — parses clean (once the
    top-level export/return, which are expected in a Workflow script, are accounted for).
  • Rendered judgePrompt({number: 75, …}) and printed the result, confirming the agent receives
    real backticks and a runnable git merge-base command rather than escape artifacts.
  • uv run pre-commit run --files .claude/skills/issue-sweep/SKILL.md — cspell, ty-check and
    pytest-test all pass; the two ruff hooks skip (no Python files in the change).

No automated tests added: the change is prompt text in a skill file, which the suite does not
cover.

Additional Notes

Found while auditing the LIF skills against a sibling project's more mature equivalents. This is
the same failure mode that project's backlog-sweep skill documented and fixed — its first
recorded wrong verdict was "an unmerged branch reported as shipped" — so the check is a known-good
one being brought over rather than a speculative hardening.

…erge-base with main

The judge prompt accepted a commit SHA as proof an open issue was already
resolved, with no check that the commit is on `main`. A bare `git log --grep`
resolves from HEAD, so a sweep run from a feature branch — or an agent that
widens an empty search with `--all` — reads unmerged work as shipped. The
skill's stated evidence bar ("a merged PR #, a commit SHA, or a named shipped
file/symbol") is satisfied by that SHA, and the sweep's output is a close-list.

Verified live: issue LIF-Initiative#722 is open while two `Issue LIF-Initiative#722: …` commits sit on
`upstream/issue-722---Translator-performance`, neither on `main`. 190 commits
repo-wide are reachable but not on `main`, and the `Issue #NNN:` commit
convention means each matches a grep for its own issue number.

Two changes, both needed. The commands are now scoped (`git log main --grep`,
`git log main -S`), which fixes the root cause independent of checkout; and any
SHA must pass `git merge-base --is-ancestor <sha> main` before it can be cited,
which covers SHAs found in a PR body or issue comment. Widening an empty search
with `--all` is explicitly forbidden — an empty result from `main` is the answer.
A failed ancestor check returns open/uncertain with the branch named, rather
than dropping the finding: in-flight work is useful, it is just not a closure.

The arbiter previously said only "investigate the same way" and inherited the
gap, so it now re-runs the check on any commit either judge cited.

Documentation-only change to the skill file; no runtime code affected.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

issue-sweep skill can report an open issue as resolved from an unmerged-branch commit

1 participant