fix(agent): deploy_change pushes branch + opens PR (no more ghost-deploy)

Strategy-agent end-to-end audit (post-PR-A1..A12) caught a critical bug
that hadn't fired yet because no agent deploy has succeeded since the
agent was built (24 trades < 50-trade safety floor, so all weekly runs
have been "no-deploy diagnostic"). The first time it tries to deploy:

  agent_writes.deploy_change does:
    1. git checkout -b agent/change-<timestamp>
    2. git add <files> && git commit
    3. git checkout main
    4. git merge agent/change-<timestamp> --no-ff   ← LOCAL only
    5. git branch -d agent/change-<timestamp>

  Step 4's merge commit lives only on the runner's local checkout of
  main. There is NO `git push`. When the runner shuts down, the merge
  commit is discarded.

  Meanwhile the workflow's "Open PR for agent code changes" step
  (added in PR #160) does:
    git add <agent-modifiable-files>
    git diff --cached --quiet  → exit deployed=false if no staged diff

  Because the agent already committed and merged locally, there are no
  staged changes. The workflow exits deployed=false. No PR is opened.
  No notification fires. The agent's changelog records status=deployed
  with a commit_sha that points at the discarded local merge commit.

  Net effect: ghost deploy. Changelog says deployed; live code is
  unchanged. compute_measured_impact 5 days later compares equal-state
  windows, reports no effect. The agent may re-attempt the same change
  on the next eligible cycle (after 14d cooling), perpetually
  ghost-deploying.

Fix: deploy_change does the push and opens the PR itself.
  - After commit on the feature branch, `git push origin <branch>
    --force-with-lease`.
  - `gh pr create` to open the PR; on failure (re-run same day; PR
    already exists), recover URL via `gh pr view`.
  - Persist PR URL to data/agent/last_pr_url.txt for the workflow's
    Telegram step to pick up.
  - DO NOT merge to local main. main is protected — the PR is the
    deploy gate, CI runs on it, merging requires the operator (or an
    auto-merge rule).

Workflow change: the "Open PR for agent code changes" step is renamed
to "Surface agent deploy PR" and rewritten to read the PR URL the
agent persisted. It no longer does its own push or pr-create — that
responsibility now lives entirely in agent_writes.deploy_change. The
"Run Strategy Agent" step gets GH_TOKEN env so the agent's gh CLI
calls authenticate.

Tests: 3 new in TestDeployChange covering the new push+PR path —
successful PR creation, gh pr create fails (PR already exists)
recovery via gh pr view, git push failure aborts cleanly.

The obsolete test_merge_failure was removed (no merge step anymore).

Verified locally: 2820 tests pass (was 2818; net +2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: costajohnt

.github/workflows/strategy-review.yml

@@ -98,6 +98,12 @@ jobs:
       # later step propagates failure to the job status.
       - name: Run Strategy Agent (Weekly Review)
         id: agent_run
+        env:
+          # gh CLI inside the agent's deploy_change uses GITHUB_TOKEN to
+          # push the agent/change-* feature branch and open the deploy PR.
+          # PR-B1: prevents the prior "ghost deploy" where deploy_change
+          # merged to local main but never pushed.
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: uv run python scripts/strategy_agent.py --mode weekly
         continue-on-error: true
 
@@ -134,69 +140,48 @@ jobs:
           # checkout failure rather than risk committing to wrong branch)
           git checkout main
 
-      # Only deploy if the agent itself ran cleanly (a crashed agent may have
-      # left partial changes on disk we don't want to PR) AND we're not in
-      # dry-run mode.
-      - name: Open PR for agent code changes
+      # PR-B1: surface the agent's deploy PR (opened by agent_writes.deploy_change
+      # via gh CLI) for the downstream Telegram notification. Pre-PR-B1 this
+      # step did the push + gh pr create itself, but the agent had already
+      # merged its work to local main, so this step saw no staged diff and
+      # exited deployed=false — the "ghost deploy" bug. Now the agent does
+      # the push+PR; this step just reads the URL the agent left in
+      # data/agent/last_pr_url.txt.
+      - name: Surface agent deploy PR
         id: agent_deploy
         if: steps.agent_run.outcome == 'success' && inputs.dry_run != true
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
-          git config user.name "strategy-agent[bot]"
-          git config user.email "strategy-agent[bot]@users.noreply.github.com"
-
-          # Stage agent-modifiable code files (ignore errors for unchanged files)
-          git add \
-            strategies/constants.py \
-            strategies/sector.py \
-            strategies/sizing.py \
-            scripts/screener_trade.py \
-            scripts/momentum_trade.py \
-            scripts/scan_with_sentiment.py \
-            2>/dev/null || true
-
-          if git diff --cached --quiet; then
+          PR_URL_FILE="data/agent/last_pr_url.txt"
+          if [ ! -f "$PR_URL_FILE" ] || [ ! -s "$PR_URL_FILE" ]; then
+            # Agent didn't deploy this run — most weekly runs don't, given
+            # safety constraints (sample size floor, deploy budget, etc.).
             echo "deployed=false" >> "$GITHUB_OUTPUT"
             exit 0
           fi
 
-          DATE=$(date -u +%Y-%m-%d)
-          BRANCH="agent/weekly-${DATE}"
-
-          # Push the staged changes to a fresh feature branch and open a PR.
-          # main is protected (4 required status checks) so direct push is rejected;
-          # the PR forces CI to pass before code reaches main.
-          git checkout -B "$BRANCH"
-          git commit -m "agent: weekly review code changes [${DATE}]"
-          git push -u origin "$BRANCH" --force-with-lease
-
-          # Diff stats for the notification step (computed against main, not HEAD~1,
-          # because the new branch may carry multiple agent commits over time).
-          git diff "origin/main..${BRANCH}" --stat > /tmp/agent-deploy-summary.txt 2>/dev/null || true
-          git diff "origin/main..${BRANCH}" --name-only > /tmp/agent-deploy-files.txt 2>/dev/null || true
-
-          # Open a PR. If a PR already exists for this branch (re-run same day),
-          # gh pr create returns non-zero; we recover the URL via gh pr view.
-          if PR_URL=$(gh pr create \
-              --base main \
-              --head "$BRANCH" \
-              --title "agent: weekly review code changes [${DATE}]" \
-              --body "Automated agent code change from the weekly Strategy Agent run. CI must pass before merge; merging deploys these parameter changes to live trading. See AGENT_LOG.md on the trading-data branch for the agent's reasoning."); then
-            echo "Opened PR: $PR_URL"
-          else
-            PR_URL=$(gh pr view "$BRANCH" --json url --jq .url 2>/dev/null || echo "unknown")
-            echo "Updated existing PR: $PR_URL"
+          PR_URL=$(cat "$PR_URL_FILE")
+          if [ -z "$PR_URL" ]; then
+            echo "deployed=false" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          echo "Agent opened PR: $PR_URL"
+
+          # Diff stats for the notification step. Pull the PR's branch from
+          # gh so we can compute against origin/main (the PR's actual diff).
+          BRANCH=$(gh pr view "$PR_URL" --json headRefName --jq .headRefName 2>/dev/null || echo "")
+          if [ -n "$BRANCH" ]; then
+            git fetch origin "$BRANCH" 2>/dev/null || true
+            git diff "origin/main..origin/$BRANCH" --stat > /tmp/agent-deploy-summary.txt 2>/dev/null || true
+            git diff "origin/main..origin/$BRANCH" --name-only > /tmp/agent-deploy-files.txt 2>/dev/null || true
           fi
 
           {
             echo "deployed=true"
             echo "pr_url=$PR_URL"
           } >> "$GITHUB_OUTPUT"
 
-          # Return to main so the AGENT_LOG step starts from a known branch.
-          git checkout main
-
       - name: Notify on agent PR opened
         if: steps.agent_deploy.outputs.deployed == 'true'
         env:

scripts/agent_writes.py

@@ -550,13 +550,16 @@ def deploy_change(commit_message: str, evidence: dict) -> dict:
     _save_changelog(changelog)
 
     # Helper to update the pending entry status in the changelog
-    def _update_pending_status(status: str, commit_sha: str | None = None, snapshot: dict | None = None):
+    def _update_pending_status(status: str, commit_sha: str | None = None,
+                                pr_url: str | None = None, snapshot: dict | None = None):
         cl = _at._load_changelog()
         for entry in cl["changes"]:
             if entry.get("id") == change_id:
                 entry["status"] = status
                 if commit_sha:
                     entry["commit_sha"] = commit_sha
+                if pr_url:
+                    entry["pr_url"] = pr_url
                 if snapshot:
                     entry["pre_deploy_snapshot"] = snapshot
                 break
@@ -591,45 +594,96 @@ def _update_pending_status(status: str, commit_sha: str | None = None, snapshot:
             ["git", "commit", "-m", f"agent: {commit_message}\n\nAutonomous strategy agent change.\nEvidence: {json.dumps(evidence, indent=2)}"],
             cwd=str(_at.PROJECT_DIR), check=True, capture_output=True,
         )
-        subprocess.run(["git", "checkout", "main"], cwd=str(_at.PROJECT_DIR), check=True, capture_output=True)
-        merge_result = subprocess.run(
-            ["git", "merge", branch, "--no-ff", "-m", f"Merge agent change: {commit_message}"],
+
+        # Capture the agent's commit SHA on the feature branch.
+        sha_result = subprocess.run(
+            ["git", "rev-parse", "HEAD"], cwd=str(_at.PROJECT_DIR), capture_output=True, text=True,
+        )
+        commit_sha = sha_result.stdout.strip()
+
+        # PR-B1: push the feature branch and open a PR via gh CLI. We do NOT
+        # merge to local main — main is protected (4 required status checks)
+        # and a local merge would only exist on the runner, never reaching
+        # origin (the prior "ghost deploy" bug). The PR is the deploy gate;
+        # CI runs on it, the operator (or an auto-merge rule) reviews and
+        # merges, and only then does the change reach live trading.
+        push_result = subprocess.run(
+            ["git", "push", "origin", branch, "--force-with-lease"],
             cwd=str(_at.PROJECT_DIR), capture_output=True, text=True,
         )
-        if merge_result.returncode != 0:
-            subprocess.run(["git", "merge", "--abort"], cwd=str(_at.PROJECT_DIR), capture_output=True)
+        if push_result.returncode != 0:
             subprocess.run(["git", "checkout", "main"], cwd=str(_at.PROJECT_DIR), capture_output=True)
+            subprocess.run(["git", "branch", "-D", branch], cwd=str(_at.PROJECT_DIR), capture_output=True)
             _update_pending_status("failed")
-            return {"success": False, "error": f"Merge failed: {merge_result.stderr}"}
-
-        # Get merge commit SHA
-        sha_result = subprocess.run(
-            ["git", "rev-parse", "HEAD"], cwd=str(_at.PROJECT_DIR), capture_output=True, text=True
+            return {"success": False, "error": f"git push failed: {push_result.stderr.strip()}"}
+
+        # Open the PR. If one already exists for this branch (re-run on same
+        # day), gh pr create fails non-zero; we recover the URL via gh pr view.
+        pr_body = (
+            f"Automated strategy-agent code change from the weekly review.\n\n"
+            f"**Evidence:**\n```json\n{json.dumps(evidence, indent=2)}\n```\n\n"
+            f"CI must pass before merge. Merging this PR deploys the change "
+            f"to live trading. See `AGENT_LOG.md` on the `trading-data` "
+            f"branch for the agent's full reasoning."
         )
-        commit_sha = sha_result.stdout.strip()
+        pr_create = subprocess.run(
+            ["gh", "pr", "create", "--base", "main", "--head", branch,
+             "--title", f"agent: {commit_message[:100]}",
+             "--body", pr_body],
+            cwd=str(_at.PROJECT_DIR), capture_output=True, text=True,
+        )
+        if pr_create.returncode == 0:
+            pr_url = pr_create.stdout.strip()
+        else:
+            pr_view = subprocess.run(
+                ["gh", "pr", "view", branch, "--json", "url", "--jq", ".url"],
+                cwd=str(_at.PROJECT_DIR), capture_output=True, text=True,
+            )
+            pr_url = pr_view.stdout.strip() if pr_view.returncode == 0 else ""
 
-        # Clean up branch
-        subprocess.run(["git", "branch", "-d", branch], cwd=str(_at.PROJECT_DIR), capture_output=True)
+        # Return to main so the agent's subsequent operations (and the
+        # workflow's downstream steps) see a known branch.
+        subprocess.run(["git", "checkout", "main"], cwd=str(_at.PROJECT_DIR), capture_output=True)
+
+        # Surface the PR URL to the workflow's notification step. Written to
+        # data/agent/ so the workflow's existing trading-data snapshot
+        # captures it alongside the changelog.
+        try:
+            pr_url_path = _at.PROJECT_DIR / "data" / "agent" / "last_pr_url.txt"
+            pr_url_path.parent.mkdir(parents=True, exist_ok=True)
+            pr_url_path.write_text(pr_url)
+        except OSError as e:
+            logger.warning("Could not persist PR URL for workflow surface: %s", e)
 
         # Capture performance snapshot for monitoring — route through _at for patchability
         perf = _at.read_performance_summary()
 
-        # Update changelog entry to "deployed" with commit SHA and snapshot
-        _update_pending_status("deployed", commit_sha=commit_sha, snapshot={
+        # Update changelog. Status remains "deployed" for backward compat
+        # with downstream consumers (compute_measured_impact); pr_url field
+        # records that the deploy is gated by PR merge. Future: add a
+        # post-merge ratify step that flips status from "pr_opened" to
+        # "deployed" once the PR actually merges.
+        _update_pending_status("deployed", commit_sha=commit_sha, pr_url=pr_url, snapshot={
             "total_signals": perf["signals"]["total"],
             "total_trades": perf["trades"]["total"],
         })
 
-        # Send deploy notification (failure should not block deploy)
+        # Send notification (failure should not block return)
         try:
             from scripts.notify import send_alert
-            send_alert("Agent Deploy", f"agent: {commit_message[:100]}")
+            send_alert(
+                "Agent Deploy: PR opened",
+                f"agent: {commit_message[:100]}\nPR: {pr_url}\n"
+                f"CI must pass and PR must be merged for the change to take effect.",
+            )
         except Exception as e:
-            logger.warning("Deploy notification failed (deploy still succeeded): %s", e)
+            logger.warning("Deploy notification failed (PR still opened): %s", e)
 
         return {
             "success": True,
             "commit_sha": commit_sha,
+            "pr_url": pr_url,
+            "branch": branch,
             "backtest": backtest_result,
             "tests": test_result["summary"],
         }