Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions .github/workflows/issue-analyze.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,11 @@ jobs:
──────────────────────────────────────────────────────────────────
1. docs/agents/rules.md — hard constraints
2. docs/agents/architecture.md — codebase layout & data flow
3. .claude/agents/bess-analyst.md — domain expertise checklist

Do NOT read `.claude/agents/bess-analyst.md`. It is the
`bess-analyst` sub-agent's own system prompt — it already has it in
full, and you never run the procedure it describes. Reading it costs
25 KB of your context on every turn and tells you nothing you act on.

──────────────────────────────────────────────────────────────────
PROCESS
Expand Down Expand Up @@ -120,6 +124,32 @@ jobs:
reading the file:line locations it cited. Do not just trust
the summary — quote the actual code.

The report does NOT pass verification if:
- it claims a CODE bug but cites no `file:line` — there is
nothing to check. A diagnosis that the real problem is an
unavailable sensor, the wrong inverter type, an HA
integration mismatch or a stale add-on version is a
legitimate root cause and needs no code citation;
- a cited location does not say what the report claims it says;
- actual behavior diverged from the plan (unplanned import,
floor breach, spike) and the report gives ONE blended verdict
instead of a separate verdict for each of:
- **P-optimality** — was the allocation optimal against the
inputs the DP actually had at the decision point, judged
WITHOUT hindsight?
- **Forecast error (P≠R)** — the Plan was optimal for its
own forecast, but that forecast differed from what was
Realized.
- **Control/execution noise** — the plan's commanded rate
was never actually achieved by the inverter.
Blending these has repeatedly produced wrong conclusions in
this repo.

A report that fails verification is an inconclusive result, not a
diagnosis. Publish what you found and what is still missing via
the IF YOU CAN'T REACH A CONCLUSION path below — do not dress it
up as a root cause.

5. **Publish the analysis by RUNNING these two commands.** Writing
the diagnosis as your reply does NOT publish it — you are running
headless, nothing is watching your output, and a final message
Expand Down Expand Up @@ -169,7 +199,7 @@ jobs:
ANALYSIS_EOF

gh issue comment ${{ github.event.issue.number }} --body-file /tmp/analysis.md
gh issue edit ${{ github.event.issue.number }} --add-label needs-human-review
gh issue edit ${{ github.event.issue.number }} --add-label needs-human-review --remove-label ready-for-analysis

Be honest — "I couldn't find the bug in the current code" is a
valid outcome. Do not invent a root cause to fill the template.
Expand Down
62 changes: 62 additions & 0 deletions scripts/quality-check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,68 @@ then
ERRORS=$((ERRORS + 1))
fi

echo ""
echo "📋 Checking bot workflow context contract..."
echo "-------------------------------------------"

# A workflow prompt must not tell its MAIN agent to read a `.claude/agents/*.md`
# file. Those files are sub-agent system prompts: the sub-agent already has one
# in full, so naming it as the main agent's REQUIRED READING loads a second copy
# into a different context window, on every turn, for an agent that never runs
# the procedure it describes.
#
# Stage 2 did exactly that with bess-analyst.md (25,681 B) and nothing consumed
# it (#654). Its six steps are: get context, identify the current problem,
# delegate, verify the cited file:line, publish, label -- no step applies a
# "domain expertise checklist". Worse, the section of that file which could
# plausibly serve as a judging standard (Separate Evidence from Claims) was
# already restated inline in the sub-agent task the same prompt passes, so the
# content was loaded three times over.
#
# The paired assertion matters as much as the removal: Stage 2 must still
# DELEGATE. Dropping the read is the saving; dropping the delegation would gut
# the stage, and the same "trim the prompt" instinct reaches for both.
if ! python3 - <<'PY_CTX'
import re, sys, pathlib

WF = pathlib.Path(".github/workflows")
bad = []

for f in sorted(set(WF.glob("*.yml")) | set(WF.glob("*.yaml"))):
text = f.read_text()
for i, line in enumerate(text.splitlines(), 1):
# Match the shape the regression actually takes: a NUMBERED entry in a
# reading list, e.g. "3. .claude/agents/bess-analyst.md — checklist".
# Deliberately narrow. A prose line that merely names the path is not
# flagged, because the prompt now carries an explicit "Do NOT read
# .claude/agents/bess-analyst.md" note and a gate that fought its own
# documentation would be worse than one with a known edge. An
# unnumbered "read <path>" instruction would slip through; that is an
# accepted narrowness, not a claim of completeness.
if re.match(r"^\s*\d+\.\s+\.claude/agents/", line):
bad.append(f"{f}:{i}: main agent told to read a sub-agent definition: {line.strip()}")

analyze = (WF / "issue-analyze.yml").read_text()
_, _, prompt_block = analyze.partition("prompt: |")
if not re.search(r"subagent_type[\s:`'\"]*bess-analyst", prompt_block):
bad.append("issue-analyze.yml: Stage 2 no longer delegates to the bess-analyst "
"sub-agent -- that delegation IS the stage")

if bad:
for b in bad:
print(f" {b}")
sys.exit(1)

print("✅ Bot workflow context contract intact (no sub-agent definition read by a "
"main agent, Stage 2 still delegates)")
PY_CTX
then
echo "❌ Bot workflow context contract violated"
echo " A sub-agent's definition is its own system prompt. Do not also list"
echo " it as the main agent's REQUIRED READING (#654)."
ERRORS=$((ERRORS + 1))
fi

echo ""
echo "📋 Checking agent context budget..."
echo "-------------------------------------------"
Expand Down
Loading