Stage 2 analyze exits success, charges $0.33–1.58, and posts nothing — 2 of 6 runs silently produced no comment or label

[bess-product-owner]

Problem

Stage 2 (issue-analyze.yml) exits success, spends money, and delivers
nothing — no comment on the issue, no label change. The workflow run is green,
is_error: false, permission_denials_count: 0, so every downstream signal says
the analysis happened. Nothing did.

It is not a hard failure and not intermittent noise: it reproduced on demand.

Measured

Six real (non-skipped) Stage 2 runs exist. Four landed, two did not:

Issue	Run	Outcome
#118, #252, #624, #627	success	comment posted, analyzed applied
#593 (2026-08-15 09:22)	success	nothing posted, label unchanged
#643 (2026-08-18 18:57)	success	nothing posted, label unchanged

Re-firing @claude-bot analyze on both reproduced it immediately:

Run	Issue	Duration	Turns	Cost	Comment posted
32173921625	#643	179 s	33	$1.58	no
32176358237	#643	54 s	9	$0.33	no
32176355892	#593	47 s	7	$0.54	no

~$2.45 burned across three runs on two issues, with zero product.

The turn counts are the tell. --max-turns is 100 and was never approached. The
workflow prompt requires reading three docs, delegating to the bess-analyst
sub-agent, independently re-reading every cited file:line, posting a structured
comment, and editing labels. Seven turns cannot be that. The agent is deciding
it is finished early and returning a final message instead of writing to the
issue.

Why it went unnoticed for 3 days

Every layer reports healthy:

• The workflow run is green — nothing to investigate in the Actions tab.
• The ready-for-analysis label stays on, which is indistinguishable from
  "nobody has run analyze yet".
• The backlog pass sees an un-analysed issue and reports triage_labels or
  nothing — it has no notion of "analyze was requested and silently failed".

On #593 the maintainer eventually wrote the diagnosis by hand (a 2302-character
comment on 2026-08-16), which is the real cost: the stage was paid for and the
work was done twice.

What is not the cause

Ruled out by direct inspection, so nobody re-derives them:

• Not permissions. permission_denials_count: 0, and the job grants
  issues: write with a GitHub App token that posts fine on the four runs that
  worked.
• Not the trigger gate. All six runs reached the Claude step; the if filter
  correctly skips PR comments and non-owner comments (a skipped run is the
  normal, healthy shape here).
• Not a missing sub-agent. .claude/agents/bess-analyst.md is in the
  checkout.
• Not --max-turns. 7–33 of 100 used.

Two prompt defects visible on inspection

Neither is proven to be the cause, but both are real and cheap to fix:

1. PROCESS has two steps numbered 3 — "Delegate the investigation" and
   "After the sub-agent reports back, independently verify". So the posting step
   is labelled 4 while being the fifth instruction, and an agent tracking "am I
   done with step 3" can satisfy the wrong one.
2. Posting is prose, not a command. Step 4 describes a comment body; step 5
   is the only literal shell line (gh issue edit …). Every step the runs did
   execute is one with a literal command in it.

Acceptance

A Stage 2 run must not be able to exit success without having written to the
issue. Whatever the fix, the check belongs after the agent, in the workflow —
a run that posted no comment and applied no label should fail the job loudly,
rather than passing and charging for it. needs-human-review exists precisely
for the honest-failure case and was never applied either.

Repro

gh issue comment 643 --body "@claude-bot analyze"   # as the repo owner
gh run view <id> --log | grep -E '"(num_turns|total_cost_usd|is_error)"'
gh issue view 643 --json labels,comments

[bess-product-owner]

Third re-fire also silent — now 5 of 9 runs

#542 was fired in the same batch and completed success at 19:23. Same result:
no comment, labels unchanged (bug only), the two comments on the issue are
both the maintainer's own @claude-bot analyze triggers.

Updated tally: 5 of 9 non-skipped Stage 2 runs have produced nothing, and
all three of the deliberate re-fires (#542, #593, #643) failed. It reproduces on
demand, which rules out a transient.

Note #542 is the cleanest case in the set — a first-ever analyze on an issue with
no prior bot involvement and no ready-for-analysis label. So the failure does
not need a duplicate trigger comment or a pre-existing label state to occur.