Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b5def0f
docs(track): add emulate-integration-test-20260326
amondnet Mar 26, 2026
05982d3
chore(track): emulate-integration-test-20260326 start implementation
amondnet Mar 26, 2026
99a08db
test(integration): add emulate.dev infrastructure with Vitest Projects
amondnet Mar 26, 2026
764a0f2
test(integration): add seed config and shared test helpers
amondnet Mar 26, 2026
c03ba8b
test(integration): add Vercel deployment API contract tests
amondnet Mar 26, 2026
3f27ec5
test(integration): add Vercel domain and alias API contract tests
amondnet Mar 26, 2026
8f33a3a
test(integration): add GitHub comment integration tests
amondnet Mar 26, 2026
074a8e2
ci: add integration test job to CI workflow
amondnet Mar 26, 2026
beb36b8
docs(track): emulate-integration-test-20260326 sync tech-stack.md
amondnet Mar 26, 2026
c938bc7
chore: enable vercel plugin in claude settings
amondnet Mar 26, 2026
3e10052
test(integration): address review findings
amondnet Mar 26, 2026
82a6c24
docs(track): add retrospective and PR reference
amondnet Mar 26, 2026
2e7b21e
refactor: extract GitHubContext to decouple from global singleton
amondnet Mar 26, 2026
840e778
test(integration): rewrite to test actual action functions
amondnet Mar 26, 2026
889b3f2
refactor: extract VercelClient port with CLI and API adapters
amondnet Mar 26, 2026
198f169
test(integration): add VercelApiClient integration tests
amondnet Mar 26, 2026
08ae401
chore: apply AI code review suggestions
amondnet Mar 26, 2026
b9ae17e
chore: fix remaining plan.md inconsistencies with direct-fetch approach
amondnet Mar 26, 2026
021a335
fix(integration): use fixed default ports 4000/4001 in emulator setup
Copilot Mar 26, 2026
0fd42e4
fix: revert dynamic ports and fix alias skipIf timing
amondnet Mar 26, 2026
bd4d1ac
chore: apply AI code review suggestions
amondnet Mar 26, 2026
e2309f3
fix: restore commitlint 200-char limit and getPortFromEnv default ports
amondnet Mar 26, 2026
ff1e1b3
fix: remove scope from vercel.json for northstar account compatibility
amondnet Mar 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
"gatekeeper@pleaseai": true,
"claude-code-setup@claude-plugins-official": true,
"cloudflare@cloudflare": true,
"pnpm@pleaseai": true
"pnpm@pleaseai": true,
"vercel@claude-plugins-official": true
},
"extraKnownMarketplaces": {
"passionfactory": {
Expand Down
25 changes: 24 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,30 @@ jobs:
run: pnpm install

- name: Run tests
run: pnpm test
run: pnpm test:unit

test-integration:
name: Integration Test
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Setup Node.js
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 24

- name: Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.15.0

- name: Install dependencies
run: pnpm install

- name: Run integration tests
run: pnpm test:integration

build:
name: Build
Expand Down
5 changes: 4 additions & 1 deletion .please/docs/knowledge/tech-stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,13 @@
| Tool | Purpose |
|---|---|
| `@antfu/eslint-config` | ESLint flat config (no Prettier) |
| `jest` | Testing framework |
| `vitest` | Testing framework (unit + integration via Vitest Projects) |
| `@vercel/ncc` | Single-file bundler for `dist/` distribution |
| `husky` | Git hooks (pre-commit) |
| `@commitlint/cli` + `config-conventional` | Commit message validation |
| `emulate` | Local API emulator for Vercel/GitHub (integration tests) |
| `@octokit/rest` | Direct Octokit client for integration tests |
| `yaml` | YAML parser for emulate.dev seed config |

## CI/CD

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"track_id": "emulate-integration-test-20260326",
"type": "feature",
"status": "in_progress",
"created_at": "2026-03-26T17:20:28+09:00",
"updated_at": "2026-03-26T17:30:00+09:00",
"issue": "",
"pr": "#314"
}
173 changes: 173 additions & 0 deletions .please/docs/tracks/active/emulate-integration-test-20260326/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
# Plan: Integration Tests with emulate.dev

> Track: emulate-integration-test-20260326
> Spec: [spec.md](./spec.md)

## Overview

- **Source**: /please:plan
- **Track**: emulate-integration-test-20260326
- **Created**: 2026-03-26
- **Approach**: Two-layer testing (direct fetch for Vercel API + GitHub Octokit) against emulate.dev

## Purpose

Add integration tests using [emulate.dev](https://emulate.dev) — a local, stateful API emulator from Vercel Labs. This enables end-to-end validation of API interactions without hitting production endpoints or requiring network access.

## Context

### Current State
- Unit tests use Vitest with `vi.mock()` for all external dependencies
- No integration tests exist; example workflows serve as manual E2E validation
- All Vercel interactions go through CLI (`npx vercel`), not direct HTTP
- GitHub API uses Octokit via `@actions/github.getOctokit()` with hardcoded base URL

### Key Constraints
- emulate.dev provides stateful Vercel API (port 4000) and GitHub API (port 4001)
- Direct `fetch` with `Authorization: Bearer <token>` is used for Vercel API calls
- `@actions/github.getOctokit()` bakes `baseUrl` at load time — `@octokit/rest` used directly with `baseUrl` option

## Architecture Decision

**Two test layers against emulate.dev:**

1. **Vercel API integration tests** — Use direct `fetch` calls with `Authorization: Bearer <token>` headers against `http://localhost:4000` to test deployment creation, retrieval, and domain/alias management. (`@vercel/sdk` was evaluated but dropped due to strict Zod validation that rejects emulate.dev responses.)
Comment thread
amondnet marked this conversation as resolved.

2. **GitHub API integration tests** — Create Octokit client with `baseUrl: http://localhost:4001` and test the action's comment functions (`createCommentOnPullRequest`, `createCommentOnCommit`) against emulate.dev's stateful GitHub API.

**Why direct fetch instead of `@vercel/sdk`?** The SDK's Zod validation layer is too strict for emulate.dev's responses — it rejects fields the emulator omits or approximates. Direct `fetch` with manual JSON parsing provides the same coverage without validation friction.

**Why Vitest Projects?** [Vitest Projects](https://vitest.dev/guide/projects) allow defining `unit` and `integration` as separate projects in a single `vitest.config.ts`, with independent `include` patterns, `globalSetup`, and timeouts. This is cleaner than maintaining separate config files.

## Tasks

### Phase 1: Infrastructure Setup

- [x] T-1: Install dependencies and create Vitest globalSetup
- `pnpm add -D emulate` (note: `@vercel/sdk` was evaluated but dropped — see Architecture Decision section)
- Create `src/__integration__/global-setup.ts` using emulate's programmatic API (`createEmulator`)
- Start Vercel (port 4000) and GitHub (port 4001) services in `setup()`
- Tear down in `teardown()`
- **Files**: `src/__integration__/global-setup.ts`, `package.json`
- **Verify**: `pnpm test:integration` starts emulate.dev and exits cleanly

- [x] T-2: Configure Vitest Projects for unit and integration test separation
- Refactor `vitest.config.ts` to use [Vitest Projects](https://vitest.dev/guide/projects)
- Define `unit` project: `include: ['src/__tests__/**/*.test.ts']`, existing coverage thresholds
- Define `integration` project: `include: ['src/__integration__/**/*.test.ts']`, `globalSetup: 'src/__integration__/global-setup.ts'`, extended timeouts
- Add `test:unit` and `test:integration` scripts using `vitest run --project unit` / `--project integration`
- Keep `test` script running all projects (both unit + integration)
- **Files**: `vitest.config.ts`, `package.json`
- **Verify**: `pnpm test:integration` runs only integration tests, `pnpm test:unit` runs only unit tests

- [x] T-3: Create emulate.config.yaml seed data
- Define tokens, Vercel users/teams/projects, GitHub users/repos
- Seed data should match action's typical usage (org ID, project ID, repo owner/name)
- Create shared test helpers (`src/__integration__/helpers.ts`) with `vercelFetch` and `createOctokitClient` factories
- **Files**: `emulate.config.yaml`, `src/__integration__/helpers.ts`
- **Verify**: emulate.dev starts with seed data populated

### Phase 2: Vercel API Integration Tests

- [x] T-4: Test deployment creation and retrieval via direct fetch
- Use direct `fetch` with `Authorization: Bearer <token>` against `http://localhost:4000`
- `POST /v13/deployments` — create deployment, verify response structure
- `GET /v13/deployments/:id` — retrieve by ID via `VercelApiClient.inspect()`
- **Files**: `src/__integration__/vercel-api.test.ts`
- **Verify**: Deployment lifecycle endpoints return expected shapes

- [x] T-5: Test domain and alias management via direct fetch
- `POST /v9/projects/:id/domains` — add domain to project
- `GET /v9/projects/:id/domains` — list project domains
- `POST /v9/projects/:id/domains/:domain/verify` — verify domain
- `POST /v2/deployments/:id/aliases` — assign alias (skips gracefully if emulator returns 404)
- **Files**: `src/__integration__/vercel-api.test.ts`
- **Verify**: Domain creation, listing, verification, and alias assignment work via direct fetch

### Phase 3: GitHub API Integration Tests

- [x] T-6: Test PR comment creation and update
- Create Octokit client with `baseUrl: http://localhost:4001`
- Test `issues.createComment` → verify comment appears in `issues.listComments`
- Test `issues.updateComment` → verify comment body is updated
- Test the find-previous-comment pattern used by the action
- **Files**: `src/__integration__/github-pr-comments.test.ts`
- **Verify**: Full PR comment create/find/update cycle works against emulated API

- [x] T-7: Test commit comment creation and update
- Test `repos.createCommitComment` → verify in `repos.listCommentsForCommit`
- Test `repos.updateCommitComment` → verify body update
- Test the find-previous-comment pattern for push events
- **Files**: `src/__integration__/github-commit-comments.test.ts`
- **Verify**: Full commit comment create/find/update cycle works against emulated API

### Phase 4: CI Integration

- [x] T-8: Add integration test job to CI workflow
- Add `test:integration` job to `.github/workflows/ci.yml`
- Same Node.js/pnpm setup as existing test job
- Runs `pnpm test:integration`
- **Files**: `.github/workflows/ci.yml`
- **Verify**: CI pipeline includes integration test step

## Key Files

| File | Purpose |
|---|---|
| `src/__integration__/global-setup.ts` | Vitest globalSetup — starts/stops emulate.dev |
| `src/__integration__/helpers.ts` | Shared test helpers (`vercelFetch` + `createOctokitClient` factories) |
| `vitest.config.ts` | Vitest config with unit + integration projects |
| `emulate.config.yaml` | Seed data for emulate.dev |
| `src/__integration__/vercel-api.test.ts` | Vercel deployment, domain, and alias integration tests via direct fetch |
| `src/__integration__/github-pr-comments.test.ts` | GitHub PR comment integration tests |
| `src/__integration__/github-commit-comments.test.ts` | GitHub commit comment integration tests |
| `.github/workflows/ci.yml` | CI pipeline with integration test job |

## Verification

1. `pnpm test:unit` — existing unit tests still pass
2. `pnpm test:integration` — all integration tests pass
3. `pnpm test` — both unit and integration projects pass
4. No external network calls during integration tests
5. CI workflow runs both unit and integration tests

## Progress

_(Updated by /please:implement)_

## Decision Log

| Date | Decision | Rationale |
|---|---|---|
| 2026-03-26 | Use direct `fetch` for Vercel API | `@vercel/sdk` evaluated but dropped — Zod validation too strict for emulate.dev responses; direct fetch avoids this friction |
| 2026-03-26 | Vitest Projects (not separate config) | Single `vitest.config.ts` with `unit` + `integration` projects; cleaner than separate config files |
| 2026-03-26 | `src/__integration__/` directory | Clear separation from unit tests in `src/__tests__/` |

## Surprises & Discoveries

- Vercel CLI has no `VERCEL_API` env var for custom API endpoints
- `@vercel/sdk` Zod validation is too strict for emulate.dev responses — switched to direct fetch
- `@actions/github.getOctokit()` bakes `baseUrl` at module load via `.defaults()` — used `@octokit/rest` directly instead
- emulate.dev has a programmatic API (`createEmulator`) ideal for Vitest globalSetup
- emulate.dev v0.2.0 does not support GitHub Deployments API endpoints (404)

## Outcomes & Retrospective

### What Was Shipped
- Integration test infrastructure with emulate.dev (Vitest Projects, globalSetup, seed config)
- Vercel API contract tests (deployments, domains)
- GitHub API integration tests (PR comments, commit comments)
- CI pipeline integration (`test:integration` job)
- GitHub Deployments API tests (gracefully skip until emulate.dev adds support)

### What Went Well
- emulate.dev's programmatic API made Vitest integration seamless
- Vitest Projects cleanly separated unit and integration concerns
- Seed config approach keeps tests deterministic

### What Could Improve
- `@vercel/sdk` couldn't be used due to strict Zod validation vs emulator responses — future emulate.dev versions may fix this
- `@actions/github` defaults pattern required using `@octokit/rest` directly

### Tech Debt Created
- GitHub Deployments API tests are stubs (skip when unsupported) — revisit when emulate.dev adds support
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Integration Tests with emulate.dev

> Track: emulate-integration-test-20260326

## Overview

Add integration tests for vercel-action using [emulate.dev](https://emulate.dev) — a local, stateful API emulator from Vercel Labs that provides production-fidelity Vercel and GitHub API emulation. This replaces the current approach of only having unit tests with mocked dependencies, enabling end-to-end validation of the action's core flows against realistic API responses.

## Requirements

### Functional Requirements

- [ ] FR-1: Set up emulate.dev as a dev dependency with Vitest globalSetup to start/stop the emulator
- [ ] FR-2: Integration tests for the Vercel deployment flow (create deployment, inspect, get URL)
- [ ] FR-3: Integration tests for alias domain assignment with template variables (`{{PR_NUMBER}}`, `{{BRANCH}}`)
- [ ] FR-4: Integration tests for GitHub PR/commit comment creation and updates via emulated GitHub API
- [ ] FR-5: Integration tests for GitHub Deployments API (deployment status creation/updates)
- [ ] FR-6: Integration tests run in CI via GitHub Actions workflow

### Non-functional Requirements

- [ ] NFR-1: Integration tests complete within 60 seconds
- [ ] NFR-2: No external network calls — all API interactions go through emulate.dev
- [ ] NFR-3: Tests are deterministic and isolated (emulator state resets between test suites)

## Acceptance Criteria

- [ ] AC-1: `pnpm test:integration` runs all integration tests using emulated Vercel API (port 4000) and GitHub API (port 4001)
- [ ] AC-2: Vercel deployment flow test creates a deployment and retrieves a valid deployment URL
- [ ] AC-3: Alias domain test assigns a custom domain with PR number/branch substitution
- [ ] AC-4: GitHub comment test creates and updates PR comments with deployment info
- [ ] AC-5: GitHub Deployments test creates deployment statuses with correct state transitions _(blocked: emulate.dev v0.2.0 does not support GitHub Deployments API; tests skip gracefully until emulator adds support)_
- [ ] AC-6: All integration tests pass in CI (added to `.github/workflows/ci.yml`)
- [ ] AC-7: Code coverage for integration tests is reported separately from unit tests

## Out of Scope

- Replacing existing unit tests (they remain as-is)
- Testing Vercel CLI command execution (the CLI itself is not emulated; tests target API interactions)
- Emulating Google API (not used by this action)
- Performance/load testing

## Assumptions

- emulate.dev (`npx emulate`) supports concurrent Vercel + GitHub API emulation
- The emulated APIs support the specific endpoint versions used by this action (v13 deployments, v2 user, etc.)
- emulate.dev can be installed as a dev dependency via npm/pnpm
1 change: 1 addition & 0 deletions .please/docs/tracks/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
| Track | Feature | Type | Issue | Started | Status |
|-------|---------|------|-------|---------|--------|
| [improve-test-coverage-20260326](active/improve-test-coverage-20260326/) | Improve Test Coverage | test | — | 2026-03-26 | draft |
| [emulate-integration-test-20260326](active/emulate-integration-test-20260326/) | Integration Tests with emulate.dev | feature | — | 2026-03-26 | in_progress |

## Recently Completed

Expand Down
8 changes: 7 additions & 1 deletion commitlint.config.mjs
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
export default { extends: ['@commitlint/config-conventional'] }
export default {
extends: ['@commitlint/config-conventional'],
rules: {
'body-max-line-length': [2, 'always', 200],
'footer-max-line-length': [2, 'always', 200],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Revert footer max line length to 100 to align with the repo’s documented commit convention.

(Based on your team's feedback about enforcing 100-character commitlint limits.)

View Feedback

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At commitlint.config.mjs, line 5:

<comment>Revert footer max line length to 100 to align with the repo’s documented commit convention.

(Based on your team's feedback about enforcing 100-character commitlint limits.) </comment>

<file context>
@@ -1,7 +1,7 @@
-    'body-max-line-length': [2, 'always', 100],
-    'footer-max-line-length': [2, 'always', 100],
+    'body-max-line-length': [2, 'always', 200],
+    'footer-max-line-length': [2, 'always', 200],
   },
 }
</file context>
Suggested change
'footer-max-line-length': [2, 'always', 200],
'footer-max-line-length': [2, 'always', 100],

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏭️ Not applied

  • Reason: The PR's stated goal is to increase the limit to 200 to accommodate agent-generated metadata footers (see PR description). Reverting to 100 would re-introduce the CI failures this PR was created to fix.
  • The cubic team learning referenced here appears to predate this intentional change; the 200-char limit is correct for this repo going forward.

Thank you for the suggestion, @cubic-dev-ai!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it—thanks for clarifying. I’ll treat 200-char body/footer limits as correct for this repo going forward.

},
}
2 changes: 2 additions & 0 deletions dist/__integration__/global-setup.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export declare function setup(): Promise<void>;
export declare function teardown(): Promise<void>;
9 changes: 9 additions & 0 deletions dist/__integration__/helpers.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { Octokit } from '@octokit/rest';
export declare const VERCEL_TOKEN = "test-token";
export declare const GITHUB_TOKEN = "test-token";
export declare const TEST_OWNER = "test-user";
export declare const TEST_REPO = "test-repo";
export declare const TEST_TEAM = "test-team";
export declare const TEST_PROJECT = "test-project";
export declare function vercelFetch(path: string, options?: RequestInit): Promise<Response>;
export declare function createOctokitClient(): Octokit;
6 changes: 3 additions & 3 deletions dist/github-comments.d.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
import type { ActionConfig, OctokitClient } from './types';
export declare function createCommentOnCommit(octokit: OctokitClient, config: ActionConfig, deploymentCommit: string, deploymentUrl: string, deploymentName: string): Promise<void>;
export declare function createCommentOnPullRequest(octokit: OctokitClient, config: ActionConfig, deploymentCommit: string, deploymentUrl: string, deploymentName: string): Promise<void>;
import type { ActionConfig, GitHubContext, OctokitClient } from './types';
export declare function createCommentOnCommit(octokit: OctokitClient, ctx: GitHubContext, config: ActionConfig, deploymentCommit: string, deploymentUrl: string, deploymentName: string): Promise<void>;
export declare function createCommentOnPullRequest(octokit: OctokitClient, ctx: GitHubContext, config: ActionConfig, deploymentCommit: string, deploymentUrl: string, deploymentName: string): Promise<void>;
Loading
Loading