Skip to content

[ci, scripts] feat: add agent skills and workflow - #71

Merged
nullnonenilNULL merged 3 commits into
baidu-baige:masterfrom
Zachary-wW:feat/add-skills
Jun 12, 2026
Merged

[ci, scripts] feat: add agent skills and workflow#71
nullnonenilNULL merged 3 commits into
baidu-baige:masterfrom
Zachary-wW:feat/add-skills

Conversation

@Zachary-wW

@Zachary-wW Zachary-wW commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Set up two-pillar agent automation for LoongForge contributors:

  1. Reusable agent skills under skills/code-review (framework-agnostic PR review prompt) and submit-pr (PR submission workflow), discoverable by Claude Code via project-level config.
  2. Two @//-comment-triggered GitHub Actions workflowsclaude-review.yml for AI PR review, internal-ci.yml for bridging public PR review with the internal build pipeline.

End-to-end verified on a fork PR: the Claude reviewer reads modified files, follows skills/code-review/SKILL.md, and posts severity-tagged inline comments on file:line plus a top-level verdict / summary / checklist.

Changes

Agent skills and Claude project config (commit 1)

  • skills/code-review/SKILL.md — framework-agnostic review prompt: 10 review areas (correctness, security, API design, performance, reliability/obs, testing, readability, docs, conventions, repo hygiene), 4-tier severity (🔴 Critical / 🟠 Major / 🟡 Minor / 🟢 Nit), structured verdict + checklist output.
  • skills/submit-pr/SKILL.md — PR submission workflow: validates branch state, derives a CI-conformant title from the diff, generates the description, gets user approval, then pushes and opens the PR via gh.
  • .claude/settings.json — allow-list a small set of read-only git and pre-commit commands so they run without per-call prompts.
  • .claude/skills symlink → ../skills makes the skills discoverable by Claude Code's project-level skill loader.
  • .gitignore — keep .claude/projects/ and .claude/credentials/ out of git while allowing settings.json and the skills symlink to be committed.
  • CLAUDE.md — new "Code Review" section directs the reviewer agent to follow skills/code-review/SKILL.md and to post each finding as an inline comment via the create_inline_comment MCP tool, reserving the top-level summary for verdict + checklist only.

Claude PR review workflow (commit 2)

  • .github/workflows/claude-review.yml@claude comment-triggered review using the upstream anthropics/claude-code-action@v1.
  • Trigger: @claude mention on issue_comment, pull_request_review_comment, pull_request_review, or issues (default phrase per upstream action).
  • Authorization gate: only OWNER / MEMBER / COLLABORATOR commenters can fire the action, with a self-author bypass for the original PR/issue author. Without this gate, drive-by commenters on a public repo could trigger paid agent runs and execute tools on the self-hosted runner.
  • Permissions: contents: read (agent reads files only), pull-requests: write + issues: write (agent posts comments), id-token: write (the action's OIDC token request), actions: read.
  • API routing: env.ANTHROPIC_BASE_URL routes Claude API traffic through the internal gateway; verified via the Claude Code CLI that the gateway supports tool use + streaming.
  • Model: Claude Sonnet 4.6 — the gateway exposes Claude models with display-name IDs (verified via GET /v1/models); the canonical hyphenated form is rejected.
  • Tools: --allowedTools enables the inline review-comment MCP tool plus gh / Read / Grep / Glob.
  • Behavior: prompt: is intentionally empty so the action runs in default agent mode — PR context and GitHub MCP tools are auto-injected. Project-wide review guidance lives in CLAUDE.md and skills/code-review/SKILL.md.
  • Progress: track_progress: true shows live checkbox progress in the bot's PR comment.

Internal CI workflow (commit 3)

  • .github/workflows/internal-ci.yml/loongforge-ci PR-comment-triggered (or manual workflow_dispatch) workflow that packages PR code together with the Loong-Megatron submodule, uploads to BOS using runner-local config, writes a CI trigger signal file, and notifies the configured Ruliu group.
  • All internal config (BOS bucket, Ruliu webhook, build trigger path) is read from the self-hosted runner's local filesystem rather than from repo secrets.

Test Plan

Skills

  • code-review and submit-pr SKILL.md files are loaded by Claude Code via the .claude/skills -> ../skills symlink and discoverable as project skills.

Claude review workflow

End-to-end verified on fork PR (Zachary-wW/LoongForge#8):

  • Triggered with @claude review this pr.
  • 14 turns, ~3 min runtime, ~$3 in API cost.
  • Agent posted 6 inline review comments on specific file:line (severity-tagged: 🔴 / 🟠 / 🟡) plus 1 top-level verdict + summary + checklist comment, exactly matching the skills/code-review/SKILL.md output contract.
  • Two earlier dry-runs uncovered (and the fixes are included in this PR):
    • prompt: must be left empty for the action to run in agent mode and inject GitHub tools.
    • id-token: write is required for the action's OIDC token request.
    • Internal gateway uses display-name model IDs, not the hyphenated canonical form.
    • mcp__github_inline_comment__create_inline_comment must be explicitly allow-listed via --allowedTools.
  • The agent's own self-review caught two security regressions (missing author_association gate, over-broad contents: write); both are fixed in commit 2 before merging.

Required Setup (admin) Before Running on This Repo

  1. Add repo secrets (Settings → Secrets and variables → Actions):
    • ANTHROPIC_API_KEY — internal oneapi-comate gateway key.
    • ANTHROPIC_BASE_URL — internal gateway base URL.
  2. Self-hosted runner with labels [self-hosted, macOS, ARM64] registered to this repo (or shared from the org). The internal API gateway is on the corp network and is not reachable from GitHub-hosted runners.
  3. No GitHub App / OAuth setup is required — the action authenticates with anthropic_api_key + the workflow's built-in GITHUB_TOKEN and OIDC.

Notes

  • Per-review cost is roughly $1–$3 (multi-turn agent vs. one-shot). Acceptable for the depth gain (full-file reading, cross-reference analysis, true inline comments) but worth flagging.
  • Workflow uses the self-hosted macOS ARM64 runner already standard for this org; internal CI similarly relies on the runner's local config.

@github-actions github-actions Bot added the ci label May 27, 2026
@Zachary-wW
Zachary-wW marked this pull request as draft May 27, 2026 03:46
@Zachary-wW
Zachary-wW force-pushed the feat/add-skills branch 2 times, most recently from b842546 to d5d1e64 Compare June 2, 2026 12:32
@Zachary-wW
Zachary-wW marked this pull request as ready for review June 8, 2026 02:56
@Zachary-wW Zachary-wW changed the title [scripts] chore: WIP add AI coding skills for development workflow [ci, scripts] feat: add agent skills and comment-triggered PR automation Jun 9, 2026
Introduce two general-purpose Claude Code skills plus the project-level
configuration that lets any contributor invoke them.

Skills (under skills/):
- code-review: framework-agnostic PR review prompt. 10 review areas
  (correctness, security, API design, performance, reliability/obs,
  testing, readability, docs, conventions, repo hygiene), 4-tier
  severity (🔴 Critical / 🟠 Major / 🟡 Minor / 🟢 Nit), structured
  verdict + checklist output. Used by the claude-review workflow.
- submit-pr: PR submission workflow. Validates branch state, derives a
  CI-conformant title from the diff, generates the description, gets
  user approval, then pushes and opens the PR via gh.

Project config:
- .claude/settings.json: allow-list a small set of read-only git and
  pre-commit commands so they run without per-call prompts.
- .claude/skills -> ../skills: makes the skills discoverable by Claude
  Code's project-level skill loader.
- .gitignore: keep .claude/projects/ and .claude/credentials/ out of
  git while allowing settings.json and the skills symlink to be
  committed.
- CLAUDE.md: add a Code Review section telling the reviewer agent to
  follow skills/code-review/SKILL.md and to post each finding as an
  inline comment via the create_inline_comment MCP tool, reserving the
  top-level summary for verdict + checklist only.
@Zachary-wW Zachary-wW changed the title [ci, scripts] feat: add agent skills and comment-triggered PR automation [ci, skills] feat: add agent skills and @claude PR review workflow Jun 10, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation enhancement New feature or request model labels Jun 10, 2026
Add a comment-triggered workflow that runs Claude Code as a PR review
agent, using the upstream anthropics/claude-code-action@v1.

How it works:
- Trigger: comments matching '@claude' on issues, PRs, PR reviews, and
  PR review comments. Default phrase per upstream action.
- Authorization gate: only OWNER / MEMBER / COLLABORATOR commenters
  can fire the action, with a self-author bypass for the original
  PR/issue author. Without this gate, drive-by commenters on a public
  repo could trigger paid agent runs and execute tools on the
  self-hosted runner.
- Permissions: contents: read (agent reads files only),
  pull-requests: write + issues: write (agent posts review comments),
  id-token: write (action's OIDC token request),
  actions: read.
- Routing: env.ANTHROPIC_BASE_URL routes Claude API traffic through
  the internal gateway. Verified end to end via the Claude Code CLI;
  the gateway supports tool use + streaming.
- Model: 'Claude Sonnet 4.6' — the gateway exposes Claude models with
  display-name IDs (verified via GET /v1/models). The canonical
  hyphenated form is rejected as 'no available channel'.
- Tools: --allowedTools enables the inline review-comment MCP tool
  plus the gh / Read / Grep / Glob tools the agent needs to inspect
  the diff and modified files.
- Behavior: prompt is intentionally empty so the action runs in its
  default agent mode (PR context + GitHub MCP tools auto-injected).
  Project-wide review guidance lives in CLAUDE.md and
  skills/code-review/SKILL.md, which the agent loads automatically.
- Progress: track_progress: true shows live checkbox progress in the
  bot's initial PR comment.

End-to-end verified on a fork PR: the agent posts severity-tagged
inline comments on file:line plus a top-level verdict / summary /
checklist, matching the SKILL.md contract.
Add a workflow triggered by '/loongforge-ci' PR comments (or manual
workflow_dispatch) that packages PR code together with the
Loong-Megatron submodule, uploads it to BOS using runner-local
configuration, writes a CI trigger signal file, and notifies the
configured Ruliu group.

This bridges the public GitHub PR review process with the internal
build / test pipeline without exposing internal endpoints in the
workflow itself — all internal config (BOS bucket, Ruliu webhook,
build trigger path) is read from the self-hosted runner's local
filesystem rather than from repo secrets.
@Zachary-wW Zachary-wW changed the title [ci, skills] feat: add agent skills and @claude PR review workflow [ci, scripts] feat: add agent skills and @claude PR review workflow Jun 11, 2026
@Zachary-wW Zachary-wW changed the title [ci, scripts] feat: add agent skills and @claude PR review workflow [ci, scripts] feat: add agent skills and workflow Jun 11, 2026
@nullnonenilNULL
nullnonenilNULL merged commit 3bba602 into baidu-baige:master Jun 12, 2026
8 of 9 checks passed
@Zachary-wW

Copy link
Copy Markdown
Collaborator Author

/loongforge-ci

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci documentation Improvements or additions to documentation enhancement New feature or request model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants