Skip to content

Compute the tier-1 analyze carve-out from issue evidence, so a stale Awaiting cannot hide a qualifying bug - #685

Merged
johanzander merged 3 commits into
mainfrom
worktree-fix-issue-681-stale-awaiting-hides-tier1
Aug 23, 2026
Merged

Compute the tier-1 analyze carve-out from issue evidence, so a stale Awaiting cannot hide a qualifying bug#685
johanzander merged 3 commits into
mainfrom
worktree-fix-issue-681-stale-awaiting-hides-tier1

Conversation

@bess-agent

Copy link
Copy Markdown
Collaborator

Summary

  • The autonomous Stage 2 carve-out now fires from the issue's own evidence rather than only on items the rhythm pass had already surfaced, so a board field left stale by grooming can no longer suppress it.
  • New autonomous_analyze action in scripts/backlog-rhythm.sh, ranked with the other Analysis actions (rank 5).
  • The same predicate suppresses park / nudge_reporter / surface_discussion for those items, so one pass cannot both un-hide a bug and tell the PO to bury it.

Root cause

From #681:

The Awaiting field is treated as authoritative by the rhythm pass, but it is a board value set by grooming and can go stale relative to the issue's actual state (labels + debug log + comments). The autonomous-spend rule's conditions are checked against the item only if the rhythm surfaces it, so a stale wait value defeats the rule.

#680 met every tier-1 criterion — bug, external reporter (ridax67), debug bundle attached, no prior @claude-bot analyze — and never fired, because its card said Awaiting: reporter and the pass suppresses every "someone else owes us" wait value.

Verifying against current code confirmed the failure is one step broader than the stale field alone: the carve-out was evaluated only against items that produced some other action, so a correctly-groomed but otherwise-quiet card was equally invisible. The live board proves it — #680's Awaiting has since been corrected to analysis, and the pre-fix script still does not surface it (see Test plan).

Fix

tier1_ready in the jq program derives the condition from evidence only:

Condition What it is evidence of
bug label triage confirmed it is a bug
ready-for-analysis label triage's own confirmation the debug log is attached
no analyzed / needs-human-review label no Stage 2 history — triage re-stamps ready-for-analysis on an edited issue, so the label alone would re-fire the $0.50–2 spend every /loop tick on items analysis has already settled
author != $owner the external-report bar
awaiting != "maintainer" the one deliberate hold — the loop cannot advance without a decision there, so the escalation stays rank 0 and the spend stands down

Two consumers share that single predicate: the autonomous_analyze emitter, and (negated) the three chase guards. Sharing it is the point — with the log in hand the ball is not with the reporter, so park / nudge_reporter / surface_discussion chasing them in the same pass would contradict the carve-out. One predicate makes that contradiction impossible rather than merely unlikely.

The no-prior-analyze check deliberately stays a PO-time step (documented in the action's detail and in .claude/skills/backlog/SKILL.md), because that is also what catches an analyze still in flight — which keeps ready-for-analysis until it completes.

Documentation

  • .claude/skills/backlog/SKILL.md — updated: the Awaiting signed table gains the carve-out note, the action table gains the autonomous_analyze row, and the Autonomous-spend section is rewritten to describe the evidence basis.
  • docs/agents/bess-knowledge.md and docs/SOFTWARE_DESIGN.md — checked, neither mentions anything this diff touches (backlog tooling, not BESS behaviour).
  • docs/agents/workflow.md — checked; its ready-for-analysis label row stays accurate as written.

Test plan

  • ./scripts/quality-check.sh passes locally — 2276 passed, 50 skipped; Black/Ruff/mypy, frontend, permission surface, bot-workflow contracts all green

  • .venv/bin/pytest -m slow passes — 554 passed, 8 skipped

  • code-review on the diff — zero findings

  • Step 8, observed against the live board. Fetched a real backlog-digest.sh snapshot and replayed it through the pre-fix and post-fix scripts:

    === BEFORE (pre-fix script, live digest) ===        === AFTER (fixed script, same digest) ===
    {"issue":409,"action":"move_card"}                  {"issue":409,"action":"move_card"}
    {"issue":409,"action":"rework_review"}              {"issue":409,"action":"rework_review"}
    {"issue":520,"action":"resume_implementation"}      {"issue":520,"action":"resume_implementation"}
    {"issue":602,"action":"move_card"}                  {"issue":602,"action":"move_card"}
                                                        {"issue":680,"action":"autonomous_analyze"}
    {"issue":683,"action":"add_card"}                   {"issue":683,"action":"add_card"}
    {"issue":683,"action":"escalated"}                  {"issue":683,"action":"escalated"}
    

    Human-readable output for the new action:

    ##680 autonomous_analyze
        why: tier-1 bug: ridax67, debug log attached, no analyzed/needs-human-review label
        do : fire Stage 2 as the PO, after confirming no prior @claude-bot analyze comment: ...
    

    Not a synthetic positive: gh issue view 680 --json comments returns exactly one comment (triage's), so there is no prior analyze and Late switching #680 genuinely still needs Stage 2 fired. Every other action is byte-identical before and after, so the change adds the missing action without perturbing the rest of the pass.

Evidence the test discriminates

Ten new tests. The whole fix reverted, then each guard clause removed individually from the fixed tree:

Mutation Result
entire backlog-rhythm.sh change reverted 5 failed, 79 passed
removed and .awaiting != "maintainer" test_a_tier1_bug_awaiting_the_maintainer_escalates_instead_of_firing FAILED — 1 failed, 83 passed
removed and (.labels | index("analyzed")) == null test_a_tier1_bug_with_stage2_history_does_not_fire FAILED — 1 failed, 83 passed
removed and (.labels | index("needs-human-review")) == null test_a_tier1_bug_with_stage2_history_does_not_fire FAILED — 1 failed, 83 passed
removed and .author != \$owner test_a_bug_opened_by_the_maintainer_does_not_fire FAILED — 1 failed, 83 passed
removed and (.labels | index("ready-for-analysis")) != null test_a_bug_still_waiting_for_its_log_does_not_fire + test_quiet_backlog_is_a_noop FAILED — 2 failed, 82 passed

Restored: tree clean, 84/84 pass. No guard in the predicate is vacuous — every one of them has a test that goes red when it is deleted, which is the property the negative "does not fire" tests would otherwise only appear to have.

Outcome-level coverage

Not a DP/intent/control-mapping change, so run_scenario_realized does not apply. The outcome pinned is the pass's emitted action set, asserted through the script's real --json output via the existing _run / _actions_for harness in backend/tests/test_backlog_rhythm.py — the tests execute backlog-rhythm.sh itself against a seeded digest (RHYTHM_DIGEST_FILE), not a reimplementation of its rules. Both halves are covered: the action appears (5 tests), and the contradicting chases disappear (3 of those same tests assert park / nudge_reporter / surface_discussion absent).

Refs #681

johanzander and others added 3 commits August 23, 2026 02:42
Recovered from a session that exited before committing. See #681.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JM9cNxMtHVvZjw84Q9u9jU
…d field

The autonomous Stage 2 carve-out was only ever evaluated against items the
rhythm pass had already surfaced as actions. Any item that produced no other
action was therefore invisible to it -- including #680, a `bug` +
`ready-for-analysis` report from an external reporter with a debug bundle
attached, which sat unanalysed because its card said `Awaiting: reporter`.
A board field that grooming can leave stale silently disabled the one path
that fires Stage 2 without a human trigger.

`tier1_ready` now derives the condition from the issue's own evidence -- the
`bug` label, the `ready-for-analysis` stamp that is triage's own confirmation
the log is attached, the absence of any Stage 2 label, and a non-maintainer
author -- and emits a dedicated `autonomous_analyze` action that no wait value
can suppress. `Awaiting: maintainer` is the deliberate exception: there the
loop is held for a decision, so the escalation stays loudest and the spend
stands down.

The same predicate negates into the `park`, `nudge_reporter` and
`surface_discussion` guards. Without that, one pass could tell the PO to chase
or bury the exact reporter whose log is already in hand -- the two actions
contradict, and sharing one predicate makes contradicting each other
impossible rather than merely unlikely.

Refs #681

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JM9cNxMtHVvZjw84Q9u9jU
@bess-agent

Copy link
Copy Markdown
Collaborator Author

@claude-bot review

@bess-manager-claude-bot bess-manager-claude-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Root cause match: Yes. #681 diagnosed the bug precisely — the autonomous Stage 2 carve-out was only ever evaluated on items the rhythm pass already surfaced, so a stale Awaiting: reporter/discussion board value silently hid a qualifying tier-1 bug (#680) from the pass entirely. The fix implements the issue's own proposed option 1: compute tier1_ready in scripts/backlog-rhythm.sh from issue evidence (bug label, ready-for-analysis label, absence of analyzed/needs-human-review, non-maintainer author, awaiting != "maintainer") and evaluate it independently, rather than depending on the item surfacing via some other rule.

I verified this empirically rather than trusting the description, by fetching the PR branch into a scratch worktree and running the raw script (no pytest available in this environment, but bash+jq are) against a synthetic digest reproducing the #680 shape (bug+ready-for-analysis, Awaiting: reporter, 0 comments):

  • On main (pre-fix): zero actions emitted for the item — confirms the bug is real, not just claimed.
  • On this branch: autonomous_analyze fires for it.
  • A second case with Awaiting: maintainer correctly stays escalated only (autonomous_analyze does not override the deliberate maintainer hold), and a discussion-awaiting tier-1 bug fires autonomous_analyze with surface_discussion correctly suppressed — matching the "one predicate, shared by the emitter and the negated chase guards" design described in the PR body.

Test coverage: Strong. Ten new tests in backend/tests/test_backlog_rhythm.py, each exercising the real script via subprocess against a seeded digest (behavior, not implementation — consistent with docs/agents/testing.md). The PR body documents a mutation sweep (each guard clause removed individually, each triggers a distinct test failure) which is exactly the "must be seen to fail without its fix" bar from docs/agents/rules.md Testing section, and is more rigorous than that bar requires.

Minimal / scope: Yes. Touches only scripts/backlog-rhythm.sh (the file that already owns "decide what's due"), its tests, .claude/skills/backlog/SKILL.md docs, and CHANGELOG.md. No new class, no new file, no unrelated cleanup. docs/agents/workflow.md's existing description of ready-for-analysis (label 156: "Debug log present, awaiting @claude-bot analyze") stays consistent with the new evidence-based logic — no doc contradiction introduced.

Rule compliance:

  • Workaround check (rules.md Debugging Protocol step 8): the fix is direct, not a route-around — tier1_ready is one predicate computed once and shared by the emitter and the three chase-guard negations, not a second construction site or an extra flag bolted onto existing rules.
  • Separation of concerns: autonomous_analyze fits backlog-rhythm.sh's existing contract (deterministic evidence comparisons that decide what's due) — it doesn't take on any new responsibility the file didn't already have.
  • jq injection: --arg owner "$owner" is passed as a bound jq variable, not interpolated into the program string — no injection surface.
  • No Optional/hasattr/getattr, no new classes, no exception-string matching — none applicable (shell + jq + pytest-style subprocess tests only).

No blockers found. Approving.

@johanzander
johanzander marked this pull request as ready for review August 23, 2026 07:47
@johanzander
johanzander merged commit c2742b8 into main Aug 23, 2026
8 checks passed
@johanzander
johanzander deleted the worktree-fix-issue-681-stale-awaiting-hides-tier1 branch August 23, 2026 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants