issue-analyze.yml

# Stage 2 — Deep Analysis (gated)
#
# Triggered manually with `@claude-bot analyze` on an issue. Reads project
# context, delegates technical investigation to the `bess-analyst` sub-agent,
# reads source code to confirm, and posts a root-cause diagnosis.
#
# Does NOT modify files or open a PR. Stage 3 (issue-fix.yml) does that.

name: Issue Deep Analysis

on:
  issue_comment:
    types: [created]

jobs:
  analyze:
    name: Deep Analysis
    if: |
      github.event.comment.user.login == github.repository_owner &&
      github.event.issue.pull_request == null &&
      contains(github.event.comment.body, '@claude-bot analyze')
    runs-on: ubuntu-latest
    permissions:
      contents: read
      issues: write

    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0

      - uses: actions/create-github-app-token@v2
        id: app-token
        with:
          app-id: ${{ secrets.CLAUDE_REVIEWER_APP_ID }}
          private-key: ${{ secrets.CLAUDE_REVIEWER_PRIVATE_KEY }}

      # Marks the point the agent started, so the verification step below can
      # tell a comment IT posted from the `@claude-bot analyze` trigger comment
      # that started this run (which is always already present).
      - name: Mark analysis start
        id: started
        run: echo "at=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"

      - uses: anthropics/claude-code-action@v1
        with:
          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
          github_token: ${{ steps.app-token.outputs.token }}
          trigger_phrase: "@claude-bot analyze"
          claude_args: "--max-turns 100 --permission-mode bypassPermissions"
          prompt: |
            You are the **Stage 2 Deep Analysis** bot for issue #${{ github.event.issue.number }} in johanzander/bess-manager.

            ──────────────────────────────────────────────────────────────────
            REQUIRED READING (read these files before any analysis)
            ──────────────────────────────────────────────────────────────────
            1. docs/agents/rules.md           — hard constraints
            2. docs/agents/architecture.md    — codebase layout & data flow

            Do NOT read `.claude/agents/bess-analyst.md`. It is the
            `bess-analyst` sub-agent's own system prompt — it already has it in
            full, and you never run the procedure it describes. Reading it costs
            25 KB of your context on every turn and tells you nothing you act on.

            ──────────────────────────────────────────────────────────────────
            PROCESS
            ──────────────────────────────────────────────────────────────────
            1. Get full issue context:
                 gh issue view ${{ github.event.issue.number }} --json title,body,labels,comments

            2. **Identify the CURRENT problem.** Long-running issues evolve.
               Read ALL comments chronologically and determine:
               - What is the user struggling with RIGHT NOW (latest comments)?
               - What problems from the original report have already been
                 resolved or are no longer relevant?
               - What is the LATEST debug bundle? (Use that one, not older ones.)
               - What version is the user running? Is it current?

               If the issue has evolved significantly, the analysis MUST focus
               on the current problem, not the original report.

            3. **Delegate the investigation to the `bess-analyst` sub-agent.**
               Use the Agent tool with `subagent_type: bess-analyst`. Pass it:
                 - The issue title and body
                 - A summary of how the issue has evolved (what's resolved,
                   what the current problem is)
                 - The LATEST debug bundle (from issue body or attachments —
                   download it). If multiple bundles exist, use the most recent.
                 - This task: "Diagnose this issue independently. The reporter's
                   explanation is a HYPOTHESIS — do not assume it is correct.

                   Step 1: Read ALL issue comments to understand what the user
                   is struggling with NOW, not just the original report. Issues
                   evolve — the current problem may be completely different.

                   Step 2: Triage the LATEST debug bundle FIRST. Check sensor
                   availability, system health, connectivity errors, and whether
                   the reporter's HA integration matches the code path BESS
                   Manager uses. Check inverter type (MIN vs SPH vs SolaX) and
                   whether the expected sensors exist. If sensors are unavailable
                   or the system has fundamental issues, that is likely the real
                   problem — not a subtle code bug.

                   Step 3: Only after triage, read the design docs and source
                   files relevant to what the triage revealed. Do NOT read the
                   entire codebase — focus on the subsystem the debug bundle
                   points to.

                   Step 4: If the reporter claims a code bug, verify that
                   (a) the error actually originates from BESS Manager code,
                   (b) the behavior isn't intentional per the design docs, and
                   (c) the debug bundle evidence supports the claim.

                   Step 5: Before finalizing, re-read the last 3-5 user
                   comments and verify your analysis addresses what the user
                   is actually asking about. If it doesn't, you analyzed a
                   stale problem — go back and fix it.

                   Report: your independent diagnosis (which may differ from
                   the reporter's), file:line references, and whether the
                   debug bundle reveals a different root cause than claimed."

            4. After the sub-agent reports back, **independently verify** by
               reading the file:line locations it cited. Do not just trust
               the summary — quote the actual code.

               The report does NOT pass verification if:
                 - it claims a CODE bug but cites no `file:line` — there is
                   nothing to check. A diagnosis that the real problem is an
                   unavailable sensor, the wrong inverter type, an HA
                   integration mismatch or a stale add-on version is a
                   legitimate root cause and needs no code citation;
                 - a cited location does not say what the report claims it says;
                 - actual behavior diverged from the plan (unplanned import,
                   floor breach, spike) and the report gives ONE blended verdict
                   instead of a separate verdict for each of:
                     - **P-optimality** — was the allocation optimal against the
                       inputs the DP actually had at the decision point, judged
                       WITHOUT hindsight?
                     - **Forecast error (P≠R)** — the Plan was optimal for its
                       own forecast, but that forecast differed from what was
                       Realized.
                     - **Control/execution noise** — the plan's commanded rate
                       was never actually achieved by the inverter.
                   Blending these has repeatedly produced wrong conclusions in
                   this repo.

               A report that fails verification is an inconclusive result, not a
               diagnosis. Publish what you found and what is still missing via
               the IF YOU CAN'T REACH A CONCLUSION path below — do not dress it
               up as a root cause.

            5. **Publish the analysis by RUNNING these two commands.** Writing
               the diagnosis as your reply does NOT publish it — you are running
               headless, nothing is watching your output, and a final message
               reaches nobody. The issue only changes if `gh` changes it.

               Write the body to a file first, then post it. Use a file rather
               than `-b` because the body contains backticks, quotes and
               newlines that do not survive shell quoting:

                 cat > /tmp/analysis.md <<'ANALYSIS_EOF'
                 ## Root cause
                 <one-paragraph plain-English explanation>

                 ## Evidence
                 - `path/to/file.py:LINE` — <quoted code excerpt>
                 - <additional file:line refs as needed>

                 ## Proposed fix
                 <high-level approach in 2-4 sentences — NO code yet>

                 ## Risks / open questions
                 <anything you couldn't determine, edge cases, or test gaps>

                 ---
                 Reply `@claude-bot fix` to implement and open a draft PR.
                 ANALYSIS_EOF

                 gh issue comment ${{ github.event.issue.number }} --body-file /tmp/analysis.md

               Check that the command succeeded. If it failed, fix the problem
               and run it again — do not proceed to step 6 with the comment
               unposted, and do not end the run having only described it.

            6. Update labels:
                 gh issue edit ${{ github.event.issue.number }} --add-label analyzed --remove-label ready-for-analysis

            ──────────────────────────────────────────────────────────────────
            IF YOU CAN'T REACH A CONCLUSION
            ──────────────────────────────────────────────────────────────────
            Publish what you found, what you read, and what specific
            information is still missing — again by RUNNING the commands, not
            by describing the outcome:

                 cat > /tmp/analysis.md <<'ANALYSIS_EOF'
                 ## Inconclusive
                 <what you read, what you ruled out, what is still missing>
                 ANALYSIS_EOF

                 gh issue comment ${{ github.event.issue.number }} --body-file /tmp/analysis.md
                 gh issue edit ${{ github.event.issue.number }} --add-label needs-human-review --remove-label ready-for-analysis

            Be honest — "I couldn't find the bug in the current code" is a
            valid outcome. Do not invent a root cause to fill the template.
            An inconclusive comment that is actually posted is worth far more
            than a confident one that is not.

            ──────────────────────────────────────────────────────────────────
            HARD CONSTRAINTS
            ──────────────────────────────────────────────────────────────────
            - DO NOT modify any files.
            - DO NOT open a PR.
            - DO NOT skip the bess-analyst delegation step — it's the whole
              point of this stage.
            - DO NOT end the run without having RUN `gh issue comment`. A
              diagnosis you only wrote out is a diagnosis nobody receives,
              and the job will fail the verification step below.
            - Quote real code. Don't paraphrase.

      # The acceptance criterion of #646: a Stage 2 run must not be able to
      # exit `success` having written nothing to the issue.
      #
      # Five of nine non-skipped runs did exactly that over three days, at
      # $0.33-1.58 each, and every signal said they were healthy -- green run,
      # is_error false, permission_denials_count 0. The `ready-for-analysis`
      # label staying put is indistinguishable from "nobody has run analyze
      # yet", so the backlog pass could not see it either. On #593 the
      # maintainer eventually wrote the diagnosis by hand: the stage was paid
      # for and the work was done twice.
      #
      # `if: always()` is load-bearing -- the whole point is to catch the run
      # that the agent step already reported as successful.
      #
      # This step ASSERTS; it deliberately does not repair. Posting the comment
      # on the agent's behalf would be a second publishing path whose only job
      # is to route around the first one failing, which masks the regression
      # instead of surfacing it (docs/agents/rules.md, Debugging Protocol
      # step 8). A loud red run is the product here.
      - name: Verify the analysis reached the issue
        if: always()
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
          ISSUE: ${{ github.event.issue.number }}
          SINCE: ${{ steps.started.outputs.at }}
        run: |
          set -euo pipefail

          posted=$(gh issue view "$ISSUE" --json comments \
            --jq "[.comments[] | select(.createdAt > \"$SINCE\")] | length")
          labels=$(gh issue view "$ISSUE" --json labels --jq '[.labels[].name] | join(",")')

          echo "Comments posted since ${SINCE}: ${posted}"
          echo "Labels now: ${labels:-<none>}"

          failed=0

          if [ "$posted" -eq 0 ]; then
            echo "::error::Stage 2 posted NO comment on issue #${ISSUE}. The run"
            echo "::error::billed for an analysis that reached nobody. Do not re-fire"
            echo "::error::blindly -- check whether the agent described the comment"
            echo "::error::instead of running 'gh issue comment' (see #646)."
            failed=1
          fi

          case ",$labels," in
            *,analyzed,*|*,needs-human-review,*) ;;
            *)
              echo "::error::Stage 2 applied neither 'analyzed' nor"
              echo "::error::'needs-human-review' to issue #${ISSUE}. Without one of"
              echo "::error::them the issue is indistinguishable from one that was"
              echo "::error::never analysed, and the backlog pass will skip it."
              failed=1
              ;;
          esac

          if [ "$failed" -eq 1 ]; then
            exit 1
          fi

          echo "✅ Analysis published: ${posted} new comment(s), labels: ${labels}"