Skip to content

ci-integration.md

Latest commit

 

History

History
130 lines (93 loc) · 3.68 KB

ci-integration.md

File metadata and controls

130 lines (93 loc) · 3.68 KB

CI Integration

AgentRC — prime your repositories for AI-assisted development.

AgentRC commands return structured output and exit codes designed for CI pipelines.

Prerequisites

  • Node.js 20+ on the runner
  • Auth token — GitHub: GITHUB_TOKEN or GH_TOKEN. Azure DevOps: AZURE_DEVOPS_PAT or AZDO_PAT.
  • Copilot CLI — required for eval (it calls the Copilot SDK). Not needed for readiness. See the VS Code Copilot Chat extension docs for installation.

readiness is a pure static analysis — it works anywhere Node runs. eval invokes AI models via the Copilot SDK, so the runner needs Copilot CLI installed and authenticated.

Readiness gate

Fail if the repo drops below a maturity level:

agentrc readiness --fail-level 3 --json

Exits with code 1 if the readiness level is below 3. The --json flag outputs a machine-readable result to stdout.

Eval gate

Fail if instruction quality drops below a pass rate:

agentrc eval agentrc.eval.json --fail-level 80 --json

Exits with code 1 if the pass rate is below 80%.

GitHub Actions

name: AgentRC checks
on: [pull_request]

jobs:
  readiness:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Check readiness
        run: npx github:microsoft/agentrc readiness --fail-level 3 --json
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Run eval
        run: npx github:microsoft/agentrc eval --fail-level 80 --json
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          # eval requires the GitHub Copilot CLI to be installed
          # and authenticated on the runner (copilot → /login)

Azure Pipelines

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: "20.x"

  - script: npx github:microsoft/agentrc readiness --fail-level 3 --json
    displayName: Check readiness
    env:
      GITHUB_TOKEN: $(GITHUB_TOKEN) # for GitHub-hosted repos
      # AZURE_DEVOPS_PAT: $(AZURE_DEVOPS_PAT)  # for Azure DevOps repos

  - script: npx github:microsoft/agentrc eval --fail-level 80 --json
    displayName: Run eval
    env:
      GITHUB_TOKEN: $(GITHUB_TOKEN)
      # AZURE_DEVOPS_PAT: $(AZURE_DEVOPS_PAT)
      # eval requires the GitHub Copilot CLI to be installed
      # and authenticated on the runner (copilot → /login)

Applying policies in CI

Use a policy to enforce org-specific standards:

npx github:microsoft/agentrc readiness --policy ./policies/strict.json --fail-level 3 --json

Any CI system

The pattern works in any CI that has Node.js:

npx github:microsoft/agentrc readiness --fail-level 3 --json
npx github:microsoft/agentrc eval --fail-level 80 --json

Both commands exit 0 on success and 1 when --fail-level is breached. Use --json to get structured output for downstream tooling.

Output format

Both commands output a CommandResult<T> envelope when --json is set:

{
  "ok": true,
  "status": "success",
  "data": { ... }
}

Status values: "success", "partial", "noop", "error". The process exit code is 0 for success and 1 when --fail-level is breached or a command error occurs.

Next steps

  • At Scale — batch processing and automated PRs across orgs
  • Policies — create org-specific readiness policies
  • Commands — full flag reference for readiness and eval
  • Concepts — understand maturity levels and pass rates