Skip to content

feat(providers): add GitHub Copilot community provider #1505

Merged
Wirasm merged 30 commits into
coleam00:devfrom
danielscholl:personal/copilot-provider
May 25, 2026
Merged

feat(providers): add GitHub Copilot community provider #1505
Wirasm merged 30 commits into
coleam00:devfrom
danielscholl:personal/copilot-provider

Conversation

@danielscholl
Copy link
Copy Markdown
Contributor

@danielscholl danielscholl commented Apr 30, 2026

Summary

  • Problem: GitHub Copilot's CLI exposes a supported JSON-RPC bridge via @github/copilot-sdk, but Archon currently has no path to it. PR feat(providers): add GitHub Copilot community provider (builtIn: false) #1351 already proposes a community-provider integration; this PR offers a smaller-scope alternative with the same goal.
  • Why it matters: Copilot CLI gives users access to gpt-5, gpt-5-mini, and Anthropic via BYOK under existing GitHub billing. Second community provider after Pi (feat(providers): add Pi community provider (@mariozechner/pi-coding-agent) #1270), validating that the seam scales.
  • What changed: New packages/providers/src/community/copilot/ wrapping @github/copilot-sdk, registered via registerCopilotProvider() with builtIn: false. ~3,500 lines across 32 files. Auth precedence flipped to safer default: copilot login wins by default; generic GH_TOKEN/GITHUB_TOKEN opt-in via useLoggedInUser: false; COPILOT_GITHUB_TOKEN always honored as Copilot-specific intent.
  • What did NOT change (scope boundary): No core/ infrastructure changes. No envOverrides UI mechanism. No default-assistant config refactor. No promotion to builtIn: true. No hooks, no fallback model, no sandbox. Pi behavior preserved byte-for-byte via re-exports. Claude/Codex paths untouched.

UX Journey

Before

  User                            Archon                     AI Client
  ────                            ──────                     ─────────
  workflow.yaml:                  load workflow
    provider: copilot ─────────▶  resolve provider
                                  ❌ UnknownProviderError
  load failure ◀───────────────

After

  User                            Archon                     Copilot CLI
  ────                            ──────                     ───────────
  $ copilot login ──────────────────────────────────────▶   GitHub OAuth
  workflow.yaml:                  load workflow
    provider: copilot ─────────▶  registry resolves
    [+] effort: high              translate node options
    [+] allowed_tools: [...]      build SessionConfig
    [+] mcp: ./mcp.json           [+] expand $VARS
    [+] skills: [...]             [+] resolve dirs
                                  spawn via SDK ─────────▶  CopilotClient
                                  stream chunks ◀────────   sendAndWait
  sees reply ◀──────────────────  forward to platform
  resume conv ─────────────────▶  reuse sessionId ───────▶  resumes session

[+] marks fields newly accepted on workflow nodes when provider: copilot.

Architecture Diagram

Before

  @archon/workflows
        │ IAgentProvider
        ▼
  @archon/providers ──┬── claude/             ─── @anthropic-ai/claude-agent-sdk
  (registry)          ├── codex/              ─── @openai/codex-sdk
                      └── community/pi/       === @mariozechner/pi-coding-agent

  registerCommunityProviders() — pi (builtIn: false)

After

  @archon/workflows
        │ IAgentProvider  (unchanged contract)
        ▼
  @archon/providers ──┬── claude/             ─── @anthropic-ai/claude-agent-sdk
  (registry)          ├── codex/              ─── @openai/codex-sdk
                      ├── community/pi/       === @mariozechner/pi-coding-agent
                      ├── [+] community/copilot/  === @github/copilot-sdk
                      └── [+] shared/         ─── skills.ts, structured-output.ts
                                                  (extracted from pi/, pi re-exports)

  registerCommunityProviders() — pi + [+] copilot (both builtIn: false)

Connection inventory:

From To Status Notes
@archon/workflows @archon/providers unchanged IAgentProvider contract unchanged
@archon/providers/community/copilot @github/copilot-sdk new Lazy-loaded via dynamic import()
@archon/providers/community/pi @archon/providers/shared new Pi delegates skills + structured-output to shared/, re-exports preserve public names
@archon/providers/community/copilot @archon/providers/shared new Same shared utilities, no duplication
@archon/core/config-loader (registry types) modified One line in SAFE_ASSISTANT_FIELDS mirrors pi: ['model']

Label Snapshot

  • Risk: risk: low
  • Size: size: M
  • Scope: core, dependencies, docs, tests
  • Module: providers:community/copilot, providers:shared, providers:registry

Change Metadata

  • Change type: feature
  • Primary scope: multi — primarily providers, with secondary touches to docs and tests. One config-loader line adds copilot: ['model'] to SAFE_ASSISTANT_FIELDS (mirrors pi: ['model'] line above it). One config-types re-export of CopilotProviderDefaults (mirrors Claude/Codex/Pi pattern).

Linked Issue

Comparison to #1351

Both PRs deliver the same capability — provider: copilot with builtIn: false through the community-provider seam. They differ in scope, auth default, and verification:

This PR #1351
Files changed 32 150
Lines +3,539 / -159 +12,088 / -980
Auth default copilot login wins; env tokens opt-in Env tokens win when set
Verified platforms macOS (live e2e) Windows
envOverrides UI mechanism Adds infra across config-loader / api / web
Default-assistant config refactor Touches db/codebases.ts, db/conversations.ts
.claude/archon/plans/ planning artifact Included

The auth-precedence flip is the most user-visible difference. GH_TOKEN / GITHUB_TOKEN are commonly set for gh CLI / clone helpers / webhooks, and classic GitHub PATs lack Copilot entitlement — picking them up automatically yields a misleading "Session was not created with authentication info" SDK error. This PR ignores them by default; COPILOT_GITHUB_TOKEN is treated as explicit Copilot intent and always wins.

The packages/providers/src/community/copilot/ implementation itself is roughly equivalent — the divergence is in what surrounds it.

Validation Evidence

bun run validate    # all 10 packages green end-to-end
  • bun run type-check — all 10 packages exit 0
  • bun run lint — 0 errors, 0 warnings (max-warnings=0)
  • bun run format:check — clean
  • bun --filter @archon/providers test — full Pi batch + Copilot batch all green
  • New tests: 7 test files in community/copilot/ (config, binary-resolver + dev variant, event-bridge, provider, provider-hardening, provider-lazy-load), 3 new auth-precedence tests in provider.test.ts (21 total), 3 new tests in registry.test.ts
  • Live end-to-end on macOS (Copilot CLI + active Copilot subscription):
    • e2e-copilot-all-nodes-smoke.yaml — 12 nodes (prompt, command, loop, bash, script bun/uv, structured output via output_format, depends_on, when, trigger_rule, $nodeId.output JSON dot-access). PASS.
    • e2e-copilot-abort.yaml — manual Ctrl-C verifies session.abort() cleanup. PASS.

Security Impact

  • New permissions/capabilities? No — Copilot runs under the same process privileges as Claude / Codex / Pi.
  • New external network calls? Yes — the Copilot CLI calls GitHub's Copilot API. Equivalent surface to other providers.
  • Secrets/tokens handling changed? Yes — auth precedence flipped to safer default. copilot login credentials win unless the user explicitly sets COPILOT_GITHUB_TOKEN (intent signal) or useLoggedInUser: false to opt into generic GH tokens. Tokens read from per-request env + process.env, never persisted by Archon.
  • File-system access scope changed? No — Copilot CLI operates under cwd like other providers. enableConfigDiscovery: false is the documented default with a trust-boundary comment in the config block.
  • Binary resolution hardening: isExecutableFile validates the resolved CLI path is a regular file with the exec bit set (POSIX), not a directory or non-executable.

Compatibility / Migration

  • Backward compatible? Yes — purely additive.
  • Config/env changes? Optional — users may add assistants.copilot.model to .archon/config.yaml and supply auth via copilot login, COPILOT_GITHUB_TOKEN, or useLoggedInUser: false to opt into GH_TOKEN/GITHUB_TOKEN.
  • Database migration needed? No.
  • Pi back-compat: pi/options-translator.ts, pi/provider.ts, and pi/event-bridge.ts re-export the extracted shared/ functions under their historical names. Existing Pi tests pass unchanged.

Human Verification

What was personally validated beyond CI:

  • Verified scenarios (live Copilot CLI on macOS, active Copilot subscription):
    • e2e-copilot-all-nodes-smoke.yaml — 12 nodes including 4 AI nodes, 3 deterministic nodes, 5 DAG-feature nodes. All passed.
    • Auth precedence regression: cleared useLoggedInUser from config, verified GH_TOKEN was correctly ignored (provider falls back to copilot login).
    • COPILOT_GITHUB_TOKEN round-trip: tokenSource telemetry reports copilot-token when set.
    • Default-empty copilot: {} config — provider falls through to model: 'auto' and useLoggedInUser: true.
  • Edge cases checked:
    • Workflow node output substitution — $structured-node.output.status (string, single-quoted by shellQuote) and $structured-node.output.value (number, unquoted) both round-trip correctly.
    • Pi back-compat — bun --filter @archon/providers test runs the full Pi batch alongside Copilot batches; all green.
  • What was not verified:
    • Live Copilot CLI on Linux or Windows — only verified on macOS. The os.platform()-aware tests + cross-platform contract of @github/copilot-sdk are the safety net.
    • Long-running sessions past the SDK's 24 h ceiling.
    • Promotion to builtIn: true and the four deferred capabilities.

Side Effects / Blast Radius

  • Affected subsystems:
    • @archon/providers — new community/copilot/ (provider, config, binary resolver, event bridge, capabilities, registration, 7 test files); new shared/ (skills + structured-output) extracted from Pi; one registry entry; one index.ts export.
    • @archon/providers Pi — three files trimmed (event-bridge.ts, options-translator.ts, provider.ts) to delegate to shared/, with re-exports preserving every public name and signature.
    • @archon/core config — one line in SAFE_ASSISTANT_FIELDS; one re-export of CopilotProviderDefaults in config-types.ts.
    • packages/docs-web — Copilot section added to AI-assistants page, ordered after Pi.
    • bun.lock — pulls in @github/copilot-sdk@^0.2.2 and transitive deps.
  • Potential unintended effects:
    • Pi behavior — primary regression vector, mitigated by exact-name re-exports + Pi test suite continuing to pass.
    • Workflow loading — workflows referencing provider: copilot now load successfully where they previously errored.
  • Guardrails: 21 tests in provider.test.ts (auth precedence: 5 cases including COPILOT_GITHUB_TOKEN intent and useLoggedInUser:false opt-in), 6 hardening tests, capability flags surfacing as warnings (not crashes) at workflow load.

Rollback Plan

  • Fast rollback: revert the merge commit on dev. Purely additive — no schema changes.
  • User-side rollback: remove provider: copilot from workflow YAML or assistants.copilot from .archon/config.yaml.
  • Observable failure symptoms: Copilot SDK auth errors include the friendly "Run \copilot login` (default), set COPILOT_GITHUB_TOKEN, or set `useLoggedInUser: false` ..."` message. Capability mismatches surface as system-chunk warnings.

Risks and Mitigations

  • Risk: @github/copilot-sdk is pre-1.0 (v0.2.2) — breaking changes possible on minor bumps.
    • Mitigation: pinned ^0.2.2. Type-check + Copilot tests gate any API-surface drift.
  • Risk: Generator abandonment without AbortSignal could leave a Copilot session holding resources up to 24 h.
    • Mitigation: the dag-executor (primary caller) always supplies an AbortSignal. Provider-hardening tests cover early-abort + cleanup paths.
  • Risk: Best-effort structured output depends on model instruction-following, not SDK enforcement.
  • Risk: enableConfigDiscovery: true would let the Copilot CLI load repo-level config outside Archon's workflow validation surface.
    • Mitigation: defaults to false. Inline config comment notes the trust boundary.
  • Risk: Cross-platform binary resolution untested on Linux/Windows.
    • Mitigation: os.platform()-aware tests cover the resolver paths; SDK's cross-platform contract is the production safety net.

Summary by CodeRabbit

  • New Features

    • Added GitHub Copilot community assistant with provider, registration, capabilities, config parsing, and structured-output/session streaming support.
    • Exposed Copilot CLI resolution utilities and package exports.
  • Workflows

    • New E2E Copilot smoke and manual abort test workflows.
  • Shared Utilities

    • Added shared structured-output helpers and a shared skills resolver.
  • Tests

    • Extensive tests for Copilot integration, binary resolution, event bridge, and shared utilities.
  • Documentation

    • Updated guides, env examples, and config docs for Copilot.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 30, 2026

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

Adds a GitHub Copilot community provider and related artifacts: provider implementation, binary resolver, event bridge, config types, package exports, docs and env examples, shared structured-output and skills utilities, E2E workflows, and many tests.

Changes

Copilot integration & shared tooling

Layer / File(s) Summary
Workflows, env example, and docs
.archon/workflows/test-workflows/*, .env.example, packages/docs-web/src/content/docs/*
Adds E2E Copilot smoke and abort workflows, documents COPILOT_GITHUB_TOKEN/COPILOT_BIN_PATH, and updates docs to describe Copilot provider fields and feature mappings.
Core config types
packages/core/src/config/config-loader.ts, packages/core/src/config/config-types.ts
Adds copilot to safe assistant defaults allowlist and re-exports CopilotProviderDefaults into core config types.
Package exports & test includes
packages/providers/package.json, packages/providers/src/index.ts
Adds package exports for ./community/copilot and ./community/copilot/binary-resolver, adds @github/copilot-sdk dependency, and re-exports Copilot API surface.
Copilot CLI binary resolver + tests
packages/providers/src/community/copilot/binary-resolver.ts, *binary-resolver*.test.ts
Implements path resolution, executability checks, vendor/autodetect/PATH probes and tests for dev/binary modes and exec-bit behaviors.
Event bridge & shared structured-output
packages/providers/src/community/copilot/event-bridge.ts, packages/providers/src/shared/structured-output.ts, *.test.ts
Adds AsyncQueue, event-to-chunk mapping, bridgeSession async generator, shared prompt augmentation and robust JSON parsing, plus tests.
Copilot provider implementation
packages/providers/src/community/copilot/provider.ts, capabilities.ts, config.ts, registration.ts, index.ts
Implements CopilotProvider, config parsing, capabilities declaration, provider registration, error classification, config/env merging, session creation/resume logic, structured-output wiring, and public exports.
Provider tests & registry updates
packages/providers/src/community/copilot/*.test.ts, packages/providers/src/registry*.ts
Adds comprehensive provider tests (behavior, hardening, lazy-load) and updates registry tests/registration to include copilot.
Shared skills resolver & Pi refactor
packages/providers/src/shared/skills.ts, shared/skills.test.ts, pi/*
Adds resolveSkillDirectories shared helper and tests; refactors Pi provider/options to re-export and use shared structured-output and skill-resolution helpers.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • coleam00/Archon#1297 — Migration of structured-output logic to a shared implementation used by these changes.
  • coleam00/Archon#1137 — Provider contract/type refactor that this PR extends by re-exporting provider defaults into core config types.
  • coleam00/Archon#1195 — Provider registry patterns referenced and extended by Copilot registration updates.

🐇 I hopped through code and docs with glee,

Copilot bits tucked in neatly,
Skills and schema, tests aligned,
A tiny rabbit left no line behind,
Hop on — the provider’s free.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 67.39% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat(providers): add GitHub Copilot community provider' accurately describes the main change—adding a new Copilot provider integration via the community-provider seam.
Description check ✅ Passed The PR description is comprehensive and well-structured, covering summary, UX journey, architecture diagrams, validation evidence, security impact, compatibility, and rollback plan. All key sections from the template are present and substantive.
Linked Issues check ✅ Passed The PR implementation fully satisfies #1115's requirement: direct Copilot SDK integration via provider: copilot with session management, streaming, auth flows (copilot login + token env vars), model support (gpt-5/gpt-5-mini), structured-output/skills, safety hardening, and test coverage.
Out of Scope Changes check ✅ Passed All changes directly support the Copilot provider goal. Core touches are minimal (one SAFE_ASSISTANT_FIELDS line, one re-export). Pi extraction into shared/ is justified by code reuse. Docs and tests align with #1115 scope boundaries.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.archon/workflows/test-workflows/e2e-copilot-all-nodes-smoke.yaml:
- Around line 7-8: The GH_TOKEN/GITHUB_TOKEN note is misleading because the
Copilot provider (packages/providers/src/community/copilot/provider.ts) only
uses those tokens when useLoggedInUser is set to false; update the comment in
.archon/workflows/.../e2e-copilot-all-nodes-smoke.yaml to either (a) state that
generic GH tokens will only be used when the provider is configured with
useLoggedInUser: false, or (b) instruct users to set COPILOT_GITHUB_TOKEN (or
explicitly set useLoggedInUser=false in the test config) so the workflow will
use the token instead of the logged-in user. Mention the provider symbol
useLoggedInUser and the COPILOT_GITHUB_TOKEN/GH_TOKEN/GITHUB_TOKEN env names to
make the intended configuration clear.

In `@packages/providers/src/community/copilot/binary-resolver.test.ts`:
- Around line 146-181: The failing tests leak the resolveFromPath spy when an
assertion throws; for each test that creates a resolveFromPathSpy (the tests
calling resolver.resolveCopilotBinaryPath()), wrap the assertion and the
subsequent mockRestore() in a try/finally block so
resolveFromPathSpy.mockRestore() is always executed, e.g. create
resolveFromPathSpy = spyOn(resolver, 'resolveFromPath')..., then do try { await
expect(...).rejects.toThrow(...); } finally { resolveFromPathSpy.mockRestore();
}, applying the same pattern to both tests that currently call mockRestore()
only on the happy path to ensure the spy is always restored.

In `@packages/providers/src/community/copilot/binary-resolver.ts`:
- Around line 131-149: The vendor and autodetect resolution currently only use
fileExists() and can return non-executable paths; update the checks in the
vendor branch (getVendorBinaryName(), getArchonHome(), COPILOT_VENDOR_DIR,
vendorBinaryPath) and the autodetect loop (getAutodetectPaths(), probePath) to
call isExecutableFile(path) instead of or in addition to fileExists(), only
logging via getLog().info({ source: 'vendor' }, 'copilot.binary_resolved') or
getLog().info({ source: 'autodetect' }, 'copilot.binary_resolved') and returning
the path when isExecutableFile(...) is true; do not return paths that exist but
are not executable (optionally log a warning via getLog() if fileExists() &&
!isExecutableFile()).
- Around line 37-45: The PATH lookup in resolveFromPath uses 'copilot.exe' for
Windows which misses npm-installed shims; change the executable variable (used
by lookupCmd and _execFileSync) so on process.platform === 'win32' it is
'copilot' (not 'copilot.exe') so the where lookup can find the .cmd/.ps1 shim;
keep using lookupCmd and _execFileSync as-is and trim/split the output the same
way.

In `@packages/providers/src/community/copilot/event-bridge.ts`:
- Around line 372-390: The terminal result chunk is being marked as an error
even when the SDK auto-recovers and assistant content was delivered; change the
logic that sets result.isError and result.errors so they are only applied when
there is an actual failure that produced no assistant content (i.e., require
errorMessage && !sawAssistantContent). In the block that builds the MessageChunk
(result) using session.sessionId and capturedTokens, wrap the assignments
result.isError = true and result.errors = [errorMessage] with a conditional that
checks !sawAssistantContent so recovered turns with assistant content are not
flagged as failed.

In `@packages/providers/src/community/copilot/provider-hardening.test.ts`:
- Around line 281-292: The test currently calls collect(gen) twice and discards
the first result, which can lose the real error when firstNext resolves; modify
the try block around await firstNext so that you capture and inspect the result
of collect(gen) immediately (e.g., store the returned object from collect(gen)
and if it contains an error assign it to primaryError) rather than calling
collect(gen) again later—update references to primaryError, firstNext,
collect(gen), and gen accordingly so the first collect result's error is
preserved.

In `@packages/providers/src/shared/skills.ts`:
- Around line 62-70: The code currently joins unvalidated skill names (rawName
from skillNames) into filesystem paths (using join with roots and checking
SKILL.md), which allows path traversal like "../foo" or "nested/name"; update
the validation in the loop over skillNames (before using join/existsSync) to
reject any rawName containing path separators or parent-segment tokens (e.g.,
path.sep, '/' or '..'), empty strings, or leading slashes, or alternatively
normalize and ensure path.basename(rawName) === rawName; only proceed to create
candidate paths and call existsSync when rawName passes this validation to
prevent escaping the intended roots.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cf356cbd-6001-4fd0-9bc8-d47c0057c320

📥 Commits

Reviewing files that changed from the base of the PR and between 2945f2e and ab00dd7.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (31)
  • .archon/workflows/test-workflows/e2e-copilot-abort.yaml
  • .archon/workflows/test-workflows/e2e-copilot-all-nodes-smoke.yaml
  • .env.example
  • packages/core/src/config/config-loader.ts
  • packages/core/src/config/config-types.ts
  • packages/docs-web/src/content/docs/getting-started/ai-assistants.md
  • packages/providers/package.json
  • packages/providers/src/community/copilot/binary-resolver-dev.test.ts
  • packages/providers/src/community/copilot/binary-resolver.test.ts
  • packages/providers/src/community/copilot/binary-resolver.ts
  • packages/providers/src/community/copilot/capabilities.ts
  • packages/providers/src/community/copilot/config.test.ts
  • packages/providers/src/community/copilot/config.ts
  • packages/providers/src/community/copilot/event-bridge.test.ts
  • packages/providers/src/community/copilot/event-bridge.ts
  • packages/providers/src/community/copilot/index.ts
  • packages/providers/src/community/copilot/provider-hardening.test.ts
  • packages/providers/src/community/copilot/provider-lazy-load.test.ts
  • packages/providers/src/community/copilot/provider.test.ts
  • packages/providers/src/community/copilot/provider.ts
  • packages/providers/src/community/copilot/registration.ts
  • packages/providers/src/community/pi/event-bridge.test.ts
  • packages/providers/src/community/pi/event-bridge.ts
  • packages/providers/src/community/pi/options-translator.ts
  • packages/providers/src/community/pi/provider.ts
  • packages/providers/src/index.ts
  • packages/providers/src/registry.test.ts
  • packages/providers/src/registry.ts
  • packages/providers/src/shared/skills.ts
  • packages/providers/src/shared/structured-output.ts
  • packages/providers/src/types.ts

Comment thread .archon/workflows/test-workflows/e2e-copilot-all-nodes-smoke.yaml Outdated
Comment thread packages/providers/src/community/copilot/binary-resolver.test.ts Outdated
Comment thread packages/providers/src/community/copilot/binary-resolver.ts
Comment thread packages/providers/src/community/copilot/binary-resolver.ts
Comment thread packages/providers/src/community/copilot/event-bridge.ts
Comment thread packages/providers/src/community/copilot/provider-hardening.test.ts
Comment thread packages/providers/src/shared/skills.ts
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
packages/providers/src/community/copilot/binary-resolver.ts (1)

116-159: ⚡ Quick win

Use guideline-compliant structured event names for resolver logs.

The resolver logs currently use copilot.binary_resolved; this doesn’t follow the repository’s standard state suffix set. Rename to a standard _completed event and emit a _failed event before throwing to keep observability consistent.

Suggested patch
-    getLog().info({ source: 'env' }, 'copilot.binary_resolved');
+    getLog().info({ source: 'env' }, 'copilot.binary_resolution_completed');
@@
-    getLog().info({ source: 'config' }, 'copilot.binary_resolved');
+    getLog().info({ source: 'config' }, 'copilot.binary_resolution_completed');
@@
-      getLog().info({ source: 'vendor' }, 'copilot.binary_resolved');
+      getLog().info({ source: 'vendor' }, 'copilot.binary_resolution_completed');
@@
-      getLog().info({ source: 'autodetect' }, 'copilot.binary_resolved');
+      getLog().info({ source: 'autodetect' }, 'copilot.binary_resolution_completed');
@@
-    getLog().info({ source: 'path' }, 'copilot.binary_resolved');
+    getLog().info({ source: 'path' }, 'copilot.binary_resolution_completed');
@@
-  throw new Error(
+  getLog().error(
+    { source: 'resolver' },
+    'copilot.binary_resolution_failed'
+  );
+  throw new Error(

As per coding guidelines, structured logging must use {domain}.{action}_{state} with standard states like _completed and _failed.

Also applies to: 162-177

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/providers/src/community/copilot/binary-resolver.ts` around lines 116
- 159, The resolver emits non-compliant event names like
'copilot.binary_resolved' in multiple spots; update all getLog().info calls in
this module (e.g., the returns after envPath, configCliPath, vendorBinaryPath,
autodetect probePath, and fromPath) to use a guideline-compliant name such as
'copilot.binary_resolve_completed', and for every place that throws an Error
(for example the configCliPath non-executable check inside the function using
isExecutableFile and any other pre-return error conditions) emit a failure event
before throwing using 'copilot.binary_resolve_failed' with context, ensuring the
same naming change is applied to the occurrences noted around lines 162-177
(e.g., where resolveFromPath is validated).
packages/providers/src/community/copilot/event-bridge.ts (1)

391-399: ⚡ Quick win

Parse structured output from the final assistant payload when available.

This block always parses assistantBuffer, even though sendAndWait() already returns the complete assistant message. If streaming delivers some deltas but misses the tail, structuredOutput is dropped even though the final payload is still parseable.

♻️ Proposed fix
     if (wantsStructured) {
-      const parsed = tryParseStructuredOutput(assistantBuffer);
+      const structuredOutputSource = sendResult?.data?.content ?? assistantBuffer;
+      const parsed = tryParseStructuredOutput(structuredOutputSource);
       if (parsed !== undefined) {
         result.structuredOutput = parsed;
       } else {
         log.warn(
-          { bufferLength: assistantBuffer.length, sessionId: session.sessionId },
+          {
+            bufferLength: structuredOutputSource.length,
+            sessionId: session.sessionId,
+          },
           'copilot.structured_output_parse_failed'
         );
       }
     }
Based on learnings: `packages/providers/src/**/*.ts` should handle session management and streaming (`for await (const event of events) { await platform.send(event) }`).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/providers/src/community/copilot/event-bridge.ts` around lines 391 -
399, The current block only attempts to parse structured output from the
incremental assistantBuffer (using wantsStructured, tryParseStructuredOutput,
assistantBuffer) and can drop structuredOutput if streaming missed the tail;
update the logic to prefer parsing the final assistant payload returned by
sendAndWait (or the equivalent finalAssistantPayload/assistantMessage object
returned by the send operation) and only fallback to parsing assistantBuffer if
that final payload is unavailable, then assign result.structuredOutput
accordingly; ensure session streaming loop handling (the for await (const event
of events) { await platform.send(event) } pattern) remains intact so the final
payload is always captured before parsing.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/providers/src/community/copilot/binary-resolver.ts`:
- Around line 116-159: The resolver emits non-compliant event names like
'copilot.binary_resolved' in multiple spots; update all getLog().info calls in
this module (e.g., the returns after envPath, configCliPath, vendorBinaryPath,
autodetect probePath, and fromPath) to use a guideline-compliant name such as
'copilot.binary_resolve_completed', and for every place that throws an Error
(for example the configCliPath non-executable check inside the function using
isExecutableFile and any other pre-return error conditions) emit a failure event
before throwing using 'copilot.binary_resolve_failed' with context, ensuring the
same naming change is applied to the occurrences noted around lines 162-177
(e.g., where resolveFromPath is validated).

In `@packages/providers/src/community/copilot/event-bridge.ts`:
- Around line 391-399: The current block only attempts to parse structured
output from the incremental assistantBuffer (using wantsStructured,
tryParseStructuredOutput, assistantBuffer) and can drop structuredOutput if
streaming missed the tail; update the logic to prefer parsing the final
assistant payload returned by sendAndWait (or the equivalent
finalAssistantPayload/assistantMessage object returned by the send operation)
and only fallback to parsing assistantBuffer if that final payload is
unavailable, then assign result.structuredOutput accordingly; ensure session
streaming loop handling (the for await (const event of events) { await
platform.send(event) } pattern) remains intact so the final payload is always
captured before parsing.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4a293a45-36b2-49d8-b0c7-c4e3621d586c

📥 Commits

Reviewing files that changed from the base of the PR and between ab00dd7 and 9994da6.

📒 Files selected for processing (6)
  • .archon/workflows/test-workflows/e2e-copilot-all-nodes-smoke.yaml
  • packages/providers/src/community/copilot/binary-resolver.test.ts
  • packages/providers/src/community/copilot/binary-resolver.ts
  • packages/providers/src/community/copilot/event-bridge.ts
  • packages/providers/src/community/copilot/provider-hardening.test.ts
  • packages/providers/src/shared/skills.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • packages/providers/src/community/copilot/provider-hardening.test.ts
  • .archon/workflows/test-workflows/e2e-copilot-all-nodes-smoke.yaml
  • packages/providers/src/shared/skills.ts
  • packages/providers/src/community/copilot/binary-resolver.test.ts

@danielscholl danielscholl changed the title feat(providers): add GitHub Copilot community provider (builtIn: false) feat(providers): add GitHub Copilot community provider Apr 30, 2026
@Wirasm
Copy link
Copy Markdown
Collaborator

Wirasm commented May 4, 2026

@danielscholl related to #1115 — GitHub Copilot community provider.

@iskandersierra
Copy link
Copy Markdown

What's missing in this PR?

@Wirasm
Copy link
Copy Markdown
Collaborator

Wirasm commented May 14, 2026

Review Summary

Verdict: minor-fixes-needed

This is a well-crafted PR — the Copilot provider is structurally sound, thoroughly tested (87 test cases across 7 files), and the shared utility extraction from Pi is clean with backward-compat re-exports. Six items need attention before merge: two syntax errors in re-export files, two missing test files for new shared modules, and three docs gaps (stale error message, missing model strings, missing Copilot config section).

Blocking issues

  • community/copilot/event-bridge.ts:163-164: Duplicate import line after the re-export — delete import { tryParseStructuredOutput } from '../../shared/structured-output';. The export alone satisfies both local use and re-export.
  • community/pi/options-translator.ts:248-249: Same duplicate-import pattern — delete the stray import { resolveSkillDirectories } from '../../shared/skills'; line. Keep the two export lines.
  • guides/authoring-workflows.md:690: The provider: validation error example lists only claude, codex, pi — add copilot.
  • guides/authoring-workflows.md:619-622: Model strings section is missing Copilot model refs (gpt-5, gpt-5-mini, claude-sonnet-4.5, auto). Add a Copilot (GitHub) bullet to the list.

Suggested fixes

  • Add shared/structured-output.test.ts: tryParseStructuredOutput and augmentPromptForJsonSchema are new public shared utilities without direct unit tests. Cover: prompt augmentation, tier-1 parse, fence stripping, forward-brace recovery, bare-primitive rejection, invalid JSON.
  • Add shared/skills.test.ts: resolveSkillDirectories needs direct tests for path-traversal guards (isAbsolute(), .., nested paths), basename !== name validation, and deduplication — the existing Pi tests exercise it transitively only.
  • reference/configuration.md: Add a new ### AI Providers -- Copilot section documenting assistants.copilot keys (after the Codex section). Also add COPILOT_GITHUB_TOKEN and COPILOT_BIN_PATH to the env var reference table, and add copilot to the DEFAULT_AI_ASSISTANT example.

Minor / nice-to-have

  • event-bridge.ts:285 and provider.ts:534: abort() and client.stop() failure handlers log at log.debug — upgrade both to log.warn.
  • package.json: The 7 new copilot test files (87 cases) aren't in the bun run test script — add them as separate bun test invocations so CI catches regressions.
  • event-bridge.ts:409: structured_output_parse_failed logs a warn but never surfaces to the user in-chat. Optional: yield a system chunk when structured output is unavailable so the user knows.
  • authoring-workflows.md: Stray period after "etc." in the Pi model strings bullet — pre-existing style inconsistency.

Compliments

  • The ProviderWarning → system-chunk pipeline for surfacing config issues is exactly the right pattern. Clean and consistent.
  • The auth precedence docs (COPILOT_GITHUB_TOKEN always wins, generic GH_TOKEN/GITHUB_TOKEN ignored) and the fork-session trade-off comment are exemplary — captures non-obvious WHY that prevents future accidental changes.
  • The Pi re-exports (resolveSkillDirectories as resolvePiSkills, etc.) ensure zero breaking changes. Excellent attention to backward compat.

Reviewed via maintainer-review-pr workflow (Pi/Minimax). Aspects run: code-review, error-handling, test-coverage, comment-quality, docs-impact.

@Wirasm
Copy link
Copy Markdown
Collaborator

Wirasm commented May 14, 2026

@danielscholl — after a deep review pass, I'm picking this PR as the canonical Copilot community provider. Closing #1351 with credit to @popemkt; you're now the single PR to land.

Before merge, here's the consolidated blocker list (all from the auto-posted maintainer-review above — none are architectural, all are addressable):

Must fix (6 HIGH)

  • event-bridge.ts:163-164 — duplicate `import` after a re-export → TS error. Delete line 164.
  • pi/options-translator.ts:248-249 — same duplicate-import pattern from the shared/skills extraction. Delete the duplicate `import { resolveSkillDirectories } from '../../shared/skills';` line.
  • Add shared/structured-output.test.ts — covering happy-path prompt augmentation, tier-1 clean parse, fence stripping, forward-brace scan recovery, bare-primitive rejection (`null`, `42`, `"string"`, `true`), and invalid JSON. The shared module is currently tested transitively via Pi's old event-bridge tests — that's coupling we want to undo.
  • Add shared/skills.test.ts — covering empty/null/undefined input, deduplication, path traversal guards (isAbsolute(), .., nested paths), and basename !== name validation.
  • guides/authoring-workflows.md:690 — stale provider-validation error message lists only `claude, codex, pi`. Add `copilot`.
  • guides/authoring-workflows.md:619-622 — model strings section omits Copilot refs (`gpt-5`, `gpt-5-mini`, `claude-sonnet-4.5`, `auto`). Add a Copilot (GitHub) bullet mirroring the Pi pattern.

Should fix (3 MEDIUM — docs reference table)

  • Add ### AI Providers -- Copilot section in reference/configuration.md, documenting model, modelReasoningEffort, copilotCliPath, configDir, enableConfigDiscovery, useLoggedUser, logLevel — mirror the Codex section.
  • Add COPILOT_GITHUB_TOKEN and COPILOT_BIN_PATH to the env var reference table (.env.example is correct; the reference docs were missed).
  • Add copilot to the DEFAULT_AI_ASSISTANT allowed-values example.

Specifically flagging one non-obvious thing

  • packages/providers/package.json — the 7 new Copilot test files (87 cases combined) aren't included in the package's test script. As-is, CI won't catch regressions in this code. Add them as separate `bun test` invocations per Archon's mock-isolation convention.

LOW findings (log-level upgrades, optional system-chunk surfacing for structured_output_parse_failed) are not blocking — address if you want or leave for a follow-up.

Credit for @popemkt

I want to be explicit: @popemkt's #1351 did the foundational work here, addressed the three concerns you raised on their PR (post-#1463 cleanup, lazy-loading SDK imports, useLoggedInUser precedence), and got real-hardware smoke evidence. Your PR builds on that — the shared/skills.ts + shared/structured-output.ts extraction is the architectural lift that makes this the canonical version, but the design lineage runs through popemkt's work too. When merging I'll include both authors in the squash commit message.

Ping me when the blockers are addressed and I'll do a final pass.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/providers/src/shared/skills.test.ts (1)

13-29: ⚡ Quick win

Add explicit return types for local test helpers.

Line 13 and Line 20 rely on inference; this violates the strict TS annotation rule used in this repo.

Suggested patch
+type FakeWorld = {
+  root: string;
+  cwd: string;
+  home: string;
+  stageSkill: (under: 'cwd' | 'home', subdir: '.agents' | '.claude', name: string) => string;
+};
+
-function makeFakeWorld() {
+function makeFakeWorld(): FakeWorld {
   const root = mkdtempSync(join(tmpdir(), 'archon-skills-test-'));
   const cwd = join(root, 'project');
   const home = join(root, 'home');
   mkdirSync(cwd, { recursive: true });
   mkdirSync(home, { recursive: true });
 
-  const stageSkill = (under: 'cwd' | 'home', subdir: '.agents' | '.claude', name: string) => {
+  const stageSkill = (
+    under: 'cwd' | 'home',
+    subdir: '.agents' | '.claude',
+    name: string,
+  ): string => {

As per coding guidelines, "**/*.{ts,tsx}: Use strict TypeScript configuration with complete type annotations for all functions; no any types without explicit justification".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/providers/src/shared/skills.test.ts` around lines 13 - 29, The
helper functions are missing explicit TypeScript return types: add a return type
annotation to makeFakeWorld (e.g. an object type with root: string, cwd: string,
home: string, and stageSkill: (...)= > string) and add an explicit function type
to the inner stageSkill (e.g. (under: 'cwd' | 'home', subdir: '.agents' |
'.claude', name: string) => string); update the signature of makeFakeWorld and
the stageSkill declaration so both have those explicit types matching their
returned values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/providers/src/shared/skills.test.ts`:
- Around line 88-93: The test titled "prefers cwd over home when the same name
exists in both" stages the home copy into the wrong directory ('.claude'), so
update the test to stage the home skill into '.agents' to match its intent;
modify the call to fake.stageSkill('home', '.claude', 'delta') to use '.agents'
instead, leaving the cwd staging (fake.stageSkill('cwd', '.agents', 'delta'))
and the assertions around resolveSkillDirectories(fake.cwd, ['delta'])
unchanged.

---

Nitpick comments:
In `@packages/providers/src/shared/skills.test.ts`:
- Around line 13-29: The helper functions are missing explicit TypeScript return
types: add a return type annotation to makeFakeWorld (e.g. an object type with
root: string, cwd: string, home: string, and stageSkill: (...)= > string) and
add an explicit function type to the inner stageSkill (e.g. (under: 'cwd' |
'home', subdir: '.agents' | '.claude', name: string) => string); update the
signature of makeFakeWorld and the stageSkill declaration so both have those
explicit types matching their returned values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 65dfbdf6-189a-4766-8839-e99d0cfb579e

📥 Commits

Reviewing files that changed from the base of the PR and between 9994da6 and bd743e8.

📒 Files selected for processing (5)
  • packages/docs-web/src/content/docs/guides/authoring-workflows.md
  • packages/docs-web/src/content/docs/reference/configuration.md
  • packages/providers/package.json
  • packages/providers/src/shared/skills.test.ts
  • packages/providers/src/shared/structured-output.test.ts
✅ Files skipped from review due to trivial changes (2)
  • packages/docs-web/src/content/docs/guides/authoring-workflows.md
  • packages/docs-web/src/content/docs/reference/configuration.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/providers/package.json

Comment thread packages/providers/src/shared/skills.test.ts
Daniel Scholl added 16 commits May 20, 2026 09:04
Define CopilotProviderDefaults with model, reasoning effort, and auth options
Include system message injection and CLI path configuration support
Implement full provider with session management, streaming, and binary resolution
Include comprehensive test coverage and lazy-load SDK pattern
Export CopilotProvider, config parser, and binary resolver utilities
Register Copilot provider in community providers initialization
Include streaming verification, token validation, and interrupt handling
Verify connectivity, output plumbing, and session management
…ments

Map Archon `max` effort to SDK `xhigh` and extend sendAndWait timeout to 60min
Handle fork-session requests with fresh session creation fallback
…el default

Add COPILOT_MODEL env var with envOverrides tracking across config system
Update provider to default model to 'auto' and enhance settings UI
Implement full Copilot SDK feature translation including tool restrictions,
session config assembly, and best-effort JSON parsing for structured output
test(copilot): cover env token precedence and override behavior
delete model-ref.ts and model-ref.test.ts
update copilot index and registration to drop isCopilotModelCompatible export
…sing

return undefined if parsed JSON is not an object
add tests covering non-object JSON in structured output parsing
implement isExecutableFile using stat/access and use it in path resolution
update errors to reference executable file and chmod guidance
export resolveFromPath and prefer PATH result when executable
- rename e2e-copilot-abort.yaml to test-workflows/e2e-copilot-abort.yaml
- add e2e-copilot-all-features.yaml and relocate smoke workflow to test-workflows
update providers to re-export shared implementations
expose shared utilities: tryParseStructuredOutput, augmentPromptForJsonSchema
update registry tests to cover copilot provider registration
verify no collision with built-ins and copilot appears in lists
Daniel Scholl and others added 12 commits May 20, 2026 09:04
update event-bridge to emit no system chunk on session.error
add provider-hardening tests for abort, trim model config and cleanup
refactor multiple files into sections for fixtures, demos, and checks
delete old e2e-copilot-smoke workflow
extend Copilot smoke tests to cover all node types and structured outputs
use DEFAULT_AI_ASSISTANT env var to select default ai assistant
update tests and docs to reflect new default and env var usage
introduce COPILOT_GITHUB_TOKEN and generic GH tokens; track tokenSource
reorder provider registration to register Pi before Copilot
use isExecutableFile for vendor and autodetect checks
validate skill names to reject absolute or traversal paths
- Add packages/providers/src/shared/structured-output.test.ts covering
  augmentPromptForJsonSchema, the happy-path clean parse, fence stripping
  (both ```json and bare ```), the forward-brace scan recovery for
  reasoning-model prose preamble, fence + preamble combo, whitespace
  trimming, invalid JSON, empty input, and the bare-primitive rejection
  contract (null/number/string/boolean).
- Add packages/providers/src/shared/skills.test.ts covering empty/null
  inputs, non-string and empty-string skipping, missing skills, cwd vs
  home resolution order, cwd-shadows-home semantics, deduplication, and
  the name-only contract (rejection of absolute paths, nested paths,
  and parent traversal). Uses a staged temp HOME so reads are isolated.
- Wire both new test files into packages/providers/package.json so they
  run in CI as separate bun test invocations.
- Add `copilot` to the registered-providers list in the validation
  error example at guides/authoring-workflows.md, add a Copilot bullet
  to the Model strings section, and add an AI Providers -- Copilot
  env-var subsection plus DEFAULT_AI_ASSISTANT enumeration to
  reference/configuration.md.

The two duplicate-import HIGH findings from the May 14 review were
hallucinations — the imports don't exist in the current branch — so
they need no fix.
- Update loadMcpConfig import to ../../mcp/config — coleam00#1459 (Codex MCP
  nodes) extracted it out of claude/provider.ts into its own module.
- Regenerate bun.lock from current dev (configVersion: 1). Old commits
  on this branch carried configVersion: 0; rebased forward unchanged
  but produced different transitive resolution on install (telegram
  markdown tests fail locally despite identical telegramify-markdown
  pin). bun install re-adds @github/copilot-sdk on top of the fresh
  lockfile.
- Stage the home copy of `delta` in `.agents` (not `.claude`) so the
  "prefers cwd over home" precedence test actually verifies precedence
  within `.agents`. Previously the home copy was in `.claude`, which
  could not have beaten the cwd `.agents` copy regardless of the
  resolver's behavior.
- Add explicit return types on `makeFakeWorld` and the inner
  `stageSkill` to satisfy the project's strict TS annotation rule.
@danielscholl danielscholl force-pushed the personal/copilot-provider branch from bd743e8 to d2de69d Compare May 20, 2026 14:28
Daniel Scholl added 2 commits May 20, 2026 09:45
- pi/event-bridge.ts: consolidate the `export-from` + `import-from`
  pair on shared/structured-output into the idiomatic
  `import { X }; export { X };` form. The preceding comment already
  promised "import once for local use and re-export" but the prior
  order said the opposite.
- authoring-workflows.md: add `copilot` to the prose listing of
  registered providers (the example validation error string below it
  already includes copilot).
coleam00#1459 (Codex MCP nodes) extracted loadMcpConfig out of
claude/provider.ts into a shared mcp/config.ts module. Update the
applyMcpServers docblock to reflect that the helper is shared, not
Claude-specific.
@danielscholl
Copy link
Copy Markdown
Contributor Author

@Wirasm — all blocker items from your list addressed (181ab43, 31694fd). Rebased onto current dev along the way (965fda6). bun run validate clean, smoke runs green. Ready for your final pass.

@Wirasm Wirasm merged commit 6ccfb4b into coleam00:dev May 25, 2026
4 checks passed
Wirasm added a commit to cropse/Archon that referenced this pull request May 25, 2026
Both PRs added a community provider, so they collide on:
- providers/registry.ts (import + register block)
- providers/registry.test.ts (per-provider describe blocks)
- docs-web ai-assistants.md (frontmatter description)
- bun.lock + bundled-defaults.generated.ts (regenerated post-merge)

Resolution: kept both providers — opencode + copilot — alongside pi.
Type-check + @archon/providers tests pass clean.

Conflicts surfaced when coleam00#1505 squash-merged into dev as 6ccfb4b.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Just a reminder to add support for the GitHub Copilot SDK

3 participants