Skip to content

feat: add 9 AI-powered GitHub Actions workflows#683

Closed
Tony363 wants to merge 2 commits intoLight-Heart-Labs:mainfrom
Tony363:feat/ai-workflows
Closed

feat: add 9 AI-powered GitHub Actions workflows#683
Tony363 wants to merge 2 commits intoLight-Heart-Labs:mainfrom
Tony363:feat/ai-workflows

Conversation

@Tony363
Copy link
Copy Markdown
Contributor

@Tony363 Tony363 commented Mar 31, 2026

Summary

Imports and adapts 9 AI-powered GitHub Actions workflows from the Ocelot project for DreamServer's Bash/Python/React architecture.

AI Code Review (PR-triggered)

  • claude-review-phase1 — Comment-only review on new PRs (~$1.50/PR)
  • claude-review-phase2 — Sensitive file detection + escalated review
  • claude-review-phase3 — Draft PR with suggested fixes

Scheduled Automation (nightly cron)

  • nightly-code-review — Daily code improvements (3 AM UTC, ~$3-8/run)
  • nightly-docs-update — Documentation sync (4 AM UTC, ~$1-3/run)
  • autonomous-code-scanner — Multi-scanner: formatting (P0), security/Bandit (P1), type hints (P2), docstrings (P3) with $100/run budget gate (2 AM UTC)

Issue & Release (event-triggered)

  • ai-issue-triage — Auto-label new issues by component and priority (~$1.50/issue)
  • issue-to-pr — Auto-implement issues as draft PRs (~$5-15/issue)
  • release-notes — AI-generated release notes on new releases (~$1.50/release)

Support Files

  • 3 prompt files (.github/prompts/) for nightly-code-review, nightly-docs-update, issue-to-pr
  • 5 Python helper scripts (.github/scripts/) for the autonomous scanner (anthropic_helper, type hint and docstring generators/appliers)
  • 3 test event JSON files (.github/test-events/) for local testing with act

Key Design Decisions

  • Anthropic API only — no AWS Bedrock, all workflows use ANTHROPIC_API_KEY
  • No multi-model consensus — removed all PAL MCP consensus gates
  • No notifications — stripped all Slack/Discord/Rube MCP steps
  • Shell-aware — added shellcheck + bash -n to guardrails and allowed tools (DreamServer is heavily Bash)
  • DreamServer protected pathsdream-server/installers/, dream-server/dream-cli, dream-server/config/ blocked from AI modifications
  • All PRs are drafts — every workflow creates draft PRs requiring human approval

Setup: Required GitHub Secrets

Only 2 secrets need to be configured in Settings → Secrets and variables → Actions:

Secret Purpose How to get it
ANTHROPIC_API_KEY All 9 workflows — powers Claude Code CLI and claude-code-action console.anthropic.com/settings/keys
PAT_TOKEN 6 workflows that create branches/PRs (phase3, scanner, nightly-review, nightly-docs, issue-to-pr, release-notes) GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Create with contents: write + pull_requests: write + issues: write scopes for this repo

GitHub Labels (already created)

The following labels were already created on the repo:

  • Component: installer, cli, dashboard, dashboard-api, extensions, docker, scripts, tests, ci-cd
  • Priority: priority:high, priority:medium, priority:low
  • Workflow: ai-generated, needs-human-review, force-review, nightly-review, issue-fix, auto-formatting, auto-security, auto-type-hints, auto-documentation

Post-Merge Verification

After merging, test each workflow category:

  1. Scheduled: Go to Actions tab → select any nightly workflow → "Run workflow" → enable dry_run → Run
  2. PR review: Open a test PR touching a few files → phases 1/2/3 should trigger
  3. Issue triage: Create a test issue → ai-issue-triage should auto-label
  4. Release notes: Create a draft release → release-notes should fire

Test plan

  • All 9 workflows pass actionlint (zero errors)
  • All 5 Python scripts pass py_compile
  • nightly-code-review preflight job — detected 12 changed files via act
  • nightly-docs-update detect-changes job — found README.md, CLAUDE.md affected via act
  • autonomous-code-scanner security-check job — found 55 Python, 42 Shell files via act
  • autonomous-code-scanner scan-formatting — reformatted 52/55 files (Docker test)
  • autonomous-code-scanner scan-security — Bandit found 18 medium findings (Docker test)
  • issue-to-pr validate job — secrets + dedup check passed via act
  • issue-to-pr guardrails — all 5 checks work: protected files, secrets, size gate, py_compile, bash -n (Docker test)
  • claude-review-phase2 detect-high-stakes — correctly identified patterns via act
  • claude-review-phase3 security-check — validated after fixing grep patterns via act
  • release-notes full dry-run — all steps resolve via act
  • No hardcoded secrets in any file
  • No leftover Ocelot/Bedrock/Slack references
  • No filename conflicts with existing CI workflows

🤖 Generated with Claude Code

Import and adapt AI automation workflows for DreamServer:

**AI Code Review (3):**
- claude-review-phase1: Comment-only PR review
- claude-review-phase2: Sensitive file detection + review
- claude-review-phase3: Draft PR with suggested fixes

**Scheduled Automation (3):**
- nightly-code-review: Daily code improvements (3 AM UTC)
- nightly-docs-update: Documentation sync (4 AM UTC)
- autonomous-code-scanner: Multi-scanner with formatting, security,
  type hints, and docstring generation (2 AM UTC)

**Issue & Release (3):**
- ai-issue-triage: Auto-label new issues
- issue-to-pr: Auto-implement issues as draft PRs
- release-notes: AI-generated release notes

Also includes:
- 3 prompt files (.github/prompts/)
- 5 helper scripts (.github/scripts/)
- 3 test event files for local testing with `act`

All workflows use Anthropic API only. Protected file patterns enforce
guardrails for dream-server/installers/, dream-server/dream-cli, and
dream-server/config/. Shell validation (shellcheck, bash -n) added
for DreamServer's Bash-heavy codebase.

Required secrets: ANTHROPIC_API_KEY, PAT_TOKEN

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@Tony363
Copy link
Copy Markdown
Contributor Author

Tony363 commented Mar 31, 2026

Post-Merge Testing Instructions

After merging, test each workflow category. All scheduled/event-triggered workflows only become available once they exist on main.


1. Required Secrets Setup

Configure these in Settings → Secrets and variables → Actions → New repository secret:

Secret Value
ANTHROPIC_API_KEY Your Anthropic API key from console.anthropic.com/settings/keys
PAT_TOKEN A fine-grained personal access token: GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens with scopes: contents: write, pull_requests: write, issues: write for this repo

2. Test Scheduled Workflows (safe — all have dry_run)

# Nightly code review — scans last 5 commits, skips PR creation
gh workflow run nightly-code-review.yml \
  --repo Light-Heart-Labs/DreamServer \
  -f dry_run=true \
  -f commits_to_analyze=5

# Nightly docs update — checks if docs need updating, skips PR creation
gh workflow run nightly-docs-update.yml \
  --repo Light-Heart-Labs/DreamServer \
  -f dry_run=true \
  -f commits_to_analyze=10

# Autonomous code scanner — runs P0+P1 only (formatting + security), skips PR creation
gh workflow run autonomous-code-scanner.yml \
  --repo Light-Heart-Labs/DreamServer \
  -f dry_run=true \
  -f skip_expensive=true

Monitor runs: gh run list --repo Light-Heart-Labs/DreamServer --limit 5


3. Test PR-Triggered Workflows

Open any new PR (or reopen this one after merge). The following should auto-trigger:

  • Phase 1 (claude-review-phase1.yml) — Posts review comments on the PR
  • Phase 2 (claude-review-phase2.yml) — Detects if sensitive files were changed, escalates review
  • Phase 3 (claude-review-phase3.yml) — Creates a draft PR with suggested fixes (blocked for fork PRs)

Note: Phase 1 uses anthropics/claude-code-action@v1 which requires ANTHROPIC_API_KEY. Phase 3 also requires PAT_TOKEN for branch/PR creation.


4. Test Issue Triage

Create a test issue:

gh issue create \
  --repo Light-Heart-Labs/DreamServer \
  --title "Test: dashboard API returns 500 on empty GPU list" \
  --body "When no GPU is detected, the /api/gpu endpoint crashes with a KeyError.

Steps to reproduce:
1. Start dashboard-api on a CPU-only machine
2. Call GET /api/gpu
3. Observe 500 error"

Expected: ai-issue-triage.yml auto-labels with bug, dashboard-api, and a priority label.

Close/delete the test issue after verifying.


5. Test Issue-to-PR

Create another test issue (the issue-to-pr.yml also triggers on issues: opened):

Warning: This will attempt to use Claude to implement the issue and create a draft PR. Requires both ANTHROPIC_API_KEY and PAT_TOKEN.

Use a small, well-scoped issue for testing. The workflow has guardrails (protected files, diff size limit of 1000 lines, secret scanning, syntax validation).


6. Test Release Notes

# Create a lightweight tag and draft release
git tag v0.0.1-test
git push origin v0.0.1-test
gh release create v0.0.1-test \
  --repo Light-Heart-Labs/DreamServer \
  --title "v0.0.1-test" \
  --notes "Test release for workflow validation" \
  --draft

Expected: release-notes.yml triggers and updates the release body with AI-generated notes.

Clean up after: gh release delete v0.0.1-test --repo Light-Heart-Labs/DreamServer -y && git push origin --delete v0.0.1-test


Expected Costs Per Test

Workflow Estimated Cost
Nightly code review (dry run) ~$3-8
Nightly docs update (dry run) ~$1-3
Autonomous scanner (P0+P1 only) ~$2.50
AI issue triage ~$1.50
Issue-to-PR ~$5-15
Release notes ~$1.50
PR review (phase 1) ~$1.50
Total for full test suite ~$16-34

- Phase 2: Add fork detection, skip claude-review and review-summary
  for fork PRs (no secrets available), add pull-requests: write permission
- Phase 3: Skip entire workflow for fork PRs via condition on
  security-check job (fork PRs can't post comments or access secrets)
- Fix test_parses_unified_apu: use get_gpu_tier() for unified APU
  name fallback when product_name is absent (was returning generic
  "AMD Radeon" instead of "Strix Halo 90+")

Co-Authored-By: Claude <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@Lightheartdevs Lightheartdevs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tony — love the ambition here. AI-powered CI is the right direction and the guardrails (protected files, secret scanning, diff size gates) are well thought out. But this needs significant rework before it can merge.

Critical — must fix

  1. All 8 third-party actions use mutable tags (@v4, @v1, @v6). None pinned to SHA. If any upstream action is compromised, ANTHROPIC_API_KEY and PAT_TOKEN are exposed. Pin every action to full commit SHA (e.g., actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11). Use Dependabot to keep them updated.

  2. Triple PR review triggering. Phases 1, 2, and 3 all trigger on pull_request: [opened, ready_for_review]. Every PR gets 3 concurrent Claude reviews at ~$2-4 each. Consolidate into a single workflow, or make Phase 3 (auto-fix) opt-in via a label like ai-fix.

  3. Recursive trigger risk. Phase 3 and the scanners create PRs using PAT_TOKEN. PRs from a PAT trigger other workflows (unlike GITHUB_TOKEN). The bot-actor guards check for claude[bot] but peter-evans/create-pull-request with a PAT shows the PAT owner as actor, not a bot. Could create infinite review loops.

  4. issue-to-pr.yml is open to any user. Anyone who opens an issue triggers Claude to write code and create a PR. Prompt injection via issue body is a real attack vector. Gate this behind a label (e.g., ai-implement) that only maintainers can apply.

Medium — should fix

  1. Secret validation anti-pattern. Multiple workflows use if [ -z "${{ secrets.X }}" ] which expands the secret into the shell script. Use env: blocks instead:
env:
  HAS_KEY: ${{ secrets.ANTHROPIC_API_KEY != '' }}
  1. gpu.py code change (+6/-1) mixed into a CI workflow PR. Split into a separate PR.

  2. release-notes.yml tool syntax. Uses Bash(git log:*) (colon) while other workflows use Bash(git log *) (space). Verify which format claude-code-action expects.

Cost concern

With all workflows active:

  • 3 nightly scheduled workflows: $9-71/day
  • Per-PR reviews (3 phases): $6-11.50 each
  • Monthly estimate at current activity: $270-2,130

For a bootstrapped project, that's significant. Consider shipping the review phase (1 consolidated workflow) and the release notes generator first. The nightly scanners and issue-to-PR can come later once you have budget headroom.

Recommendation

Split this into smaller PRs:

  1. PR A: AI code review (single consolidated phase) + release notes — ship first
  2. PR B: Nightly scanners — ship when budget allows
  3. PR C: Issue-to-PR — ship after label-gating and injection hardening

The foundation is solid. Just needs tightening before it touches production.

@Tony363
Copy link
Copy Markdown
Contributor Author

Tony363 commented Mar 31, 2026

Splitting this PR into 3 smaller PRs per review feedback from @Lightheartdevs:

  1. PR A: AI code review (phases 1-3) + issue triage + release notes — shipping first
  2. PR B: Nightly scanners (code review, docs update, autonomous scanner) — ship when budget allows
  3. PR C: Issue-to-PR — shipping after label-gating and injection hardening

All 7 review items addressed in the split PRs:

  • All actions pinned to SHA + Dependabot added
  • Phase 3 label-gated to ai-fix (no triple-trigger)
  • PAT_TOKEN replaced with GITHUB_TOKEN (prevents recursive loops)
  • issue-to-pr label-gated to ai-implement + input sanitization
  • Secret validation anti-pattern fixed (env-block pattern)
  • gpu.py separated into its own commit
  • Tool syntax standardized (Bash(git log *) + --allowedTools)

Closing in favor of the split PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants