Address 7 high-priority codebase issues

1. Structured logging: Replace 929 print() calls with stdlib logging
   via strategies/log.py factory. Two-tier: full conversion for
   operational modules, WARNING/ERROR only for display scripts.

2. Pipeline failure alerting: Telegram notifications on daily pipeline
   failures (daily_run.sh + GHA), agent deploy/rollback events.

3. CI data branch: Move trading data (signals.csv, trades.csv, agent/)
   to dedicated trading-data branch. Main stays clean for code changes.

4. agent_tools.py split: 1,702 → 3 files (164 + 686 + 959 lines).
   Safety constants + dispatch in agent_tools.py (no-touch file),
   reads in agent_reads.py, writes in agent_writes.py.

5. dashboard.py split: 2,071 → 3 files (208 + 357 + 1,516 lines).
   Routes in dashboard.py, data layer in dashboard_data.py,
   HTML/CSS/JS template extracted to templates/dashboard.html.

6. Coverage threshold: 54% minimum with branch coverage enforced in CI.

7. Drift monitor: New scripts/drift_monitor.py compares backtest
   predictions to live performance, alerts on significant deviation.
   Integrated into weekly strategy-review workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: costajohnt

.github/workflows/ci.yml

@@ -38,7 +38,7 @@ jobs:
         run: uv sync
 
       - name: Run tests with coverage
-        run: uv run python -m pytest tests/ -v --tb=short --cov=strategies --cov=scripts --cov-report=term-missing --cov-report=html
+        run: uv run python -m pytest tests/ -v --tb=short --cov=strategies --cov=scripts --cov-report=term-missing --cov-report=html --cov-fail-under=54
 
       - name: Upload coverage report
         if: always()

.github/workflows/daily-trading.yml

@@ -43,6 +43,11 @@ jobs:
       - name: Ensure data directory exists
         run: mkdir -p data
 
+      - name: Restore state from trading-data branch
+        run: |
+          git fetch origin trading-data 2>/dev/null || true
+          git checkout origin/trading-data -- data/ 2>/dev/null || echo "No prior trading data"
+
       - name: 1. Position Manager (manage exits)
         id: position_manager
         run: uv run python scripts/position_manager.py --execute --check-sentiment
@@ -77,12 +82,40 @@ jobs:
         run: uv run python scripts/strategy_agent.py --mode event-check
         continue-on-error: true
 
-      - name: Commit updated state files
+      - name: Commit state to trading-data branch
         if: always()
         run: |
           git config user.name "github-actions[bot]"
           git config user.email "github-actions[bot]@users.noreply.github.com"
+
+          # Save data directory
+          cp -r data/ /tmp/trading-data-snapshot/
+
+          # Switch to trading-data branch (create if needed)
+          git fetch origin trading-data 2>/dev/null || true
+          git checkout trading-data 2>/dev/null || git checkout --orphan trading-data
+
+          # Clean and restore data
+          rm -rf data/
+          cp -r /tmp/trading-data-snapshot/ data/
+
           git add data/
-          git diff --cached --quiet || git commit -m "chore: update trading state"
-          git pull --rebase || true
-          git push
+          git diff --cached --quiet || git commit -m "chore: update trading state [$(date -u +%Y-%m-%dT%H:%M:%SZ)]"
+          git push origin trading-data
+
+          # Return to original branch for any subsequent steps
+          git checkout ${{ github.ref_name }} 2>/dev/null || true
+
+      - name: Alert on pipeline failures
+        if: failure()
+        env:
+          TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
+          TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
+          GH_REPO: ${{ github.repository }}
+          GH_RUN_ID: ${{ github.run_id }}
+        run: |
+          if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
+            curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
+              -d chat_id="$TELEGRAM_CHAT_ID" \
+              -d text="GHA Pipeline Alert: daily-trading job failed. Check: https://github.com/${GH_REPO}/actions/runs/${GH_RUN_ID}"
+          fi

.github/workflows/strategy-review.yml

@@ -42,6 +42,10 @@ jobs:
         run: uv run python scripts/strategy_agent.py --mode weekly
         continue-on-error: true
 
+      - name: Run drift monitor
+        run: uv run python scripts/drift_monitor.py --alert
+        continue-on-error: true
+
       - name: Commit agent state and code changes
         if: always()
         run: |

.gitignore

@@ -12,6 +12,11 @@ data/trading_state.db
 data/metrics.json
 data/audit/
 
+# Trading state — lives on trading-data branch
+data/signals.csv
+data/trades.csv
+data/agent/
+
 # Test / coverage artifacts
 .coverage
 htmlcov/

CLAUDE.md

@@ -29,6 +29,7 @@ Automated equities trading system using Alpaca's API.
 - `risk_guard.py` — Portfolio-level kill switch (daily loss + drawdown from HWM)
 - `state_db.py` — SQLite persistence for all trading state (HWMs, cooldowns, partial profits, equity)
 - `weekly_trend.py` — Weekly SMA trend filter (blocks severe downtrends only)
+- `log.py` — Structured logging factory (`get_logger` with `alpaca.` namespace prefix)
 
 ### Scripts (`scripts/`)
 
@@ -38,7 +39,7 @@ Automated equities trading system using Alpaca's API.
 - `position_manager.py` — Exit manager: max-loss stop, dynamic trailing stop, partial profit, breakeven stop, cooldown tracking, null-price alerting
 - `momentum_trade.py` — Auto-trades momentum picks with momentum-adapted position sizing
 - `portfolio.py` — Account dashboard (CLI) + equity curve snapshot recording
-- `dashboard.py` — Live web dashboard (Flask, port 8050)
+- `dashboard.py` — Live web dashboard routes (Flask, port 8050)
 - `performance.py` — Signal accuracy analysis + strategy attribution + slippage analysis
 - `trade_logger.py` — CSV logging to `data/signals.csv` and `data/trades.csv` (with strategy_source column)
 - `notify.py` — macOS + Telegram notifications
@@ -51,7 +52,11 @@ Automated equities trading system using Alpaca's API.
 - `run_strategy.py` — Run single-ticker mean-reversion analysis with optional execution
 - `momentum_screener.py` — Market-wide momentum screener (volume + 10d/50d momentum)
 - `strategy_agent.py` — Autonomous strategy improvement agent (OpenRouter via OpenAI SDK, Opus 4.6, weekly + event-triggered)
-- `agent_tools.py` — 16 agent tools with safety constraints (no-touch files, change budget, cooling periods, atomic writes)
+- `agent_tools.py` — Agent safety constants, tool dispatch, and TOOL_DEFINITIONS (no-touch file)
+- `agent_reads.py` — Agent read/compute tools (side-effect-free observations)
+- `agent_writes.py` — Agent write/deploy tools (state-mutating operations)
+- `dashboard_data.py` — Dashboard data loading and performance computation
+- `drift_monitor.py` — Backtest-to-live drift detection with alerting
 - `shadow_evaluator.py` — Shadow experiment evaluator (retrospective A/B testing against real trades)
 - `daily_metrics.py` — Daily performance metrics snapshot
 - `walkforward.py` — Walk-forward backtesting validation
@@ -98,6 +103,11 @@ Automated equities trading system using Alpaca's API.
 - Shadow experiments: retrospective A/B testing of parameter changes against real trade data (max 2 active, 14-day duration)
 - All scripts use `uv run python scripts/<name>.py`
 - Run tests with: `uv run pytest tests/`
+- Structured logging: use `from strategies.log import get_logger; logger = get_logger(__name__)` — never bare `print()` for warnings/errors
+- Coverage threshold: 54% minimum with branch coverage enforced in CI (strategies/ at 78%, scripts/ lower due to CLI entry points)
+- Trading data (signals.csv, trades.csv, agent/) committed to `trading-data` branch, not `main`
+- Pipeline failure alerting: Telegram notifications on daily pipeline failures, agent deploys, and rollbacks
+- Drift monitoring: weekly comparison of backtest predictions vs live performance (>15pp win rate, >0.5 Sharpe, >2x drawdown triggers alert)
 
 ## Dependencies