diff --git a/.bito.yaml b/.bito.yaml new file mode 100644 index 000000000..28d1a0653 --- /dev/null +++ b/.bito.yaml @@ -0,0 +1,17 @@ +suggestion_mode: comprehensive +post_description: true +post_changelist: true +exclude_files: 'package-lock.json' +exclude_draft_pr: false +secret_scanner_feedback: true +linters_feedback: true +repo_level_guidelines_enabled: true +sequence_diagram_enabled: true +custom_guidelines: + general: + - name: 'Review Posture' + path: './.bito/guidelines/review-posture.txt' + - name: 'Repo Truth And Alignment' + path: './.bito/guidelines/repo-truth-and-boundaries.txt' + - name: 'Domain Invariants' + path: './.bito/guidelines/domain-invariants.txt' diff --git a/.bito/guidelines/domain-invariants.txt b/.bito/guidelines/domain-invariants.txt new file mode 100644 index 000000000..0016af7b9 --- /dev/null +++ b/.bito/guidelines/domain-invariants.txt @@ -0,0 +1,43 @@ +Critical technical invariants for the Contentful CLI: + +Build and runtime: +- TypeScript compiles to dist/ via tsc. The bin/contentful.js entrypoint requires + dist/lib/cli.js — never lib/cli.ts directly. Any change to lib/ requires a + rebuild before testing the binary. +- Module system is CommonJS. Do not introduce ESM-only imports or top-level await. +- Yargs is pinned at ~13.3.2. Do not upgrade without a dedicated migration. + +Satellite package alignment: +- contentful-management.js, contentful-migration, contentful-import, + contentful-export, and contentful-batch-libs must be version-aligned. A major + bump to one typically requires coordinated bumps across all of them. +- Some call sites use createPlainClient() in "legacy" mode. These are transitional + from the CMA v12 migration — do not remove without verifying the new API surface. + +Testing: +- Unit tests require CONTENTFUL_INTEGRATION_TEST_CMA_TOKEN and CLI_E2E_ORG_ID + environment variables to be set, even though they shouldn't need them. This is a + known coupling — do not add more dependencies on these vars in unit tests. +- Integration tests use talkback proxy recordings. When API responses change, + recordings need updating. Run tests without the proxy first, then re-record. +- E2E tests run against standalone binaries — changes to pkg config or binary + bundling should be validated with npm run test:e2e. + +Security and supply chain: +- .npmrc sets ignore-scripts=true. After npm ci, npx allow-scripts must run. + New packages that need install scripts must be added to the lavamoat.allowScripts + allowlist in package.json. +- Secrets in CI come from HashiCorp Vault, not GitHub Secrets. + +Release: +- Commit messages drive releases via semantic-release. Breaking changes MUST use + the BREAKING CHANGE: footer keyword — without it, the major bump won't trigger. +- The beta branch publishes to the npm beta channel. Do not merge experimental + work directly to main. + +Command interface stability: +- Command names, flag names, and output formats are a public API. Changes to these + affect downstream scripts and CI pipelines. Treat them as breaking changes unless + the change is purely additive. +- The .contentfulrc.json config file format is also part of the public contract. + Changes to config keys or resolution order affect all users. diff --git a/.bito/guidelines/repo-truth-and-boundaries.txt b/.bito/guidelines/repo-truth-and-boundaries.txt new file mode 100644 index 000000000..272c95d38 --- /dev/null +++ b/.bito/guidelines/repo-truth-and-boundaries.txt @@ -0,0 +1,18 @@ +Use the repository's written documentation as review context and check whether +the change matches the documented intent. + +- Start from README.md, ARCHITECTURE.md, AGENTS.md, CONTRIBUTING.md, and docs/ADRs/ + for architectural context. +- Check whether code, tests, and documentation all tell the same story. Flag + mismatches between implementation and the documented architecture or ADRs. +- Treat AGENTS.md as the authoritative guide for sharp edges and invariants. If a + change violates an invariant documented there, flag it. +- If CI or another required check already enforces a merge rule, do not ask for + duplicate PR template sections or manual checklists. +- Ask for an ADR update when a change is architecture-significant (new command, + new dependency, new integration, changed module system, framework upgrade). +- Distinguish the CLI's public command interface from internal implementation. + Public-facing changes (command names, flags, output format, exit codes) require + extra scrutiny — they are consumed by scripts and CI pipelines. +- The docs/ directory contains per-command usage documentation. If a command's + behavior changes, the corresponding docs/ file should be updated. diff --git a/.bito/guidelines/review-posture.txt b/.bito/guidelines/review-posture.txt new file mode 100644 index 000000000..9c76cf3ff --- /dev/null +++ b/.bito/guidelines/review-posture.txt @@ -0,0 +1,18 @@ +Review this pull request like the tech lead of the Contentful CLI project. + +- Prefer a few high-signal findings to a long list of minor or style-only comments. +- Prefer behavior, contract, runtime, and documentation issues over process-only + suggestions. Do not ask for duplicate PR template sections, checklists, or manual + validation acknowledgements when CI or required checks already enforce that policy. +- Keep feedback actionable: explain why it matters, how it would surface in practice, + and the clearest next step. +- If a concern is only a risk or assumption rather than a confirmed bug, say that + clearly and explain what evidence would confirm it. +- If you find no issues, say so explicitly and call out any residual uncertainty + that still deserves human attention. +- Pay special attention to breaking changes in command interfaces — the CLI is a + public API surface used in CI/CD pipelines and scripts. Flag any change to + command names, argument names, output formats, or exit codes. +- Watch for compatibility with the satellite package ecosystem (contentful-migration, + contentful-import, contentful-export, contentful-batch-libs) — version mismatches + between these packages and contentful-management.js are a recurring pain point. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..3d5f6be9b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,63 @@ + +# Agent Guide + +Read this file first. It tells you where to find context in this repo. + +## Quick Reference + +| What you need | Where to look | +|---|---| +| How this repo is structured | [ARCHITECTURE.md](./ARCHITECTURE.md) | +| How to build/test/run | [CONTRIBUTING.md](./CONTRIBUTING.md) | +| Why decisions were made | [docs/ADRs/](./docs/ADRs/) | +| What this repo does | [README.md](./README.md) | +| Per-command usage docs | [docs/](./docs/) | +| Active specs/work | [docs/specs/](./docs/specs/) | +| PR review rules | [.bito/guidelines/](./.bito/guidelines/) | +| Sensitive files & guidance | [CONTRIBUTING.md — File-Level Guidance](./CONTRIBUTING.md#file-level-guidance) | +| Operational knowledge | [ARCHITECTURE.md — Operational Knowledge](./ARCHITECTURE.md#operational-knowledge) | +| Domain concepts | [ARCHITECTURE.md — Domain Concepts](./ARCHITECTURE.md#domain-concepts) | +| CLI entrypoint | `bin/contentful.js` → `dist/lib/cli.js` | +| Command implementations | `lib/cmds/` and `lib/cmds/_cmds/` | +| Shared utilities | `lib/utils/` | +| Configuration handling | `lib/config.js` and `lib/context.js` | + +## Sharp Edges & Invariants + +- **Build before run.** The CLI requires `npm run tsc` before it can execute — `bin/contentful.js` requires `dist/lib/cli.js` which is compiled output. +- **Mixed JS/TS codebase.** Older commands are `.js`, newer ones are `.ts`. Both coexist via `allowJs: true`. Do not convert existing `.js` files to `.ts` without deliberate migration scope. +- **Yargs 13.x is pinned.** The CLI uses yargs ~13.3.2, which is significantly behind current (17.x). Do not upgrade without a dedicated migration effort — command definitions will break. +- **`ignore-scripts=true` in `.npmrc`.** After `npm ci`, you must run `npx allow-scripts` to let approved packages execute their install scripts. CI workflows already do this. +- **Unit tests need integration env vars.** Even unit tests require `CONTENTFUL_INTEGRATION_TEST_CMA_TOKEN` and `CLI_E2E_ORG_ID` to be set, due to a known coupling. +- **Commit convention is enforced by semantic-release.** A `feat:` commit triggers a minor release, `fix:` a patch. Breaking changes MUST include `BREAKING CHANGE:` in the commit footer — the `{ "breaking": true }` rule in `package.json` maps this to a major release. +- **Satellite packages must be version-aligned.** `contentful-migration`, `contentful-import`, `contentful-export`, and `contentful-batch-libs` are tightly coupled to the CMA.js version. Bumping one usually means bumping all of them. +- **`createPlainClient()` legacy mode.** Some call sites use the "legacy" deprecated CMA client mode as a transitional measure from the CMA v12 migration. These should be migrated over time but must not be removed without verifying the new API surface covers the use case. + +## Key Conventions + +- **Commit format:** Angular Conventional Commits (`type(scope): description`) via semantic-release +- **Branch strategy:** `main` (production) + `beta` (pre-release channel), squash merge +- **Test location:** `test/unit/` mirrors `lib/`, `test/integration/` uses talkback proxy, `test/e2e/` tests binaries +- **Module system:** CommonJS (compiled from TypeScript via `tsc`) +- **Command pattern:** Files in `lib/cmds/` export yargs command objects; subcommands go in `lib/cmds/_cmds/` + +## Integration Points + +**Upstream (this repo consumes):** +- **Contentful Management API** — all CRUD operations via `contentful-management.js` (^12.2.0) +- **contentful-migration** (^5.0.0) — migration script execution +- **contentful-import** (v10.0.0) / **contentful-export** (v8.0.0) — space import/export +- **contentful-batch-libs** (^11.0.0) — batch processing utilities +- **Contentful OAuth Page** — browser-based token acquisition during login + +**Downstream (consumes this repo):** +- **Developers** — `npm install -g contentful-cli` or `npx contentful-cli` +- **CI/CD pipelines** — migration execution, environment management +- **Standalone binary users** — macOS/Linux/Windows binaries from GitHub releases + +## Build & Quality + +```bash +# Quick verification loop +npm ci && npx allow-scripts && npm run tsc && npm test +``` diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..9a98bb2c9 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,132 @@ + +# Architecture + +## Overview + +`contentful-cli` is the official command-line interface for [Contentful](https://www.contentful.com). It provides subcommands for managing spaces, environments, content types, extensions, migrations, import/export, organization security checks, and the merge workflow. It is published to npm and also distributed as standalone binaries for macOS, Linux, and Windows. + +## System Context + +```mermaid +graph TD + User[Developer / CI Pipeline] --> CLI[contentful-cli] + CLI --> CMA[Contentful Management API] + CLI --> OAuth[Contentful OAuth Page] + CLI -->|delegates| Migration[contentful-migration] + CLI -->|delegates| Import[contentful-import] + CLI -->|delegates| Export[contentful-export] + CLI -->|delegates| BatchLibs[contentful-batch-libs] + CLI --> CMAjs[contentful-management.js] + CMAjs --> CMA +``` + +The CLI is the **user-facing entry point** for content-as-code workflows. It delegates heavy lifting to satellite packages (`contentful-migration`, `contentful-import`, `contentful-export`, `contentful-batch-libs`) and uses `contentful-management.js` (CMA.js) as the API client. + +## Internal Structure + +| Directory | Purpose | +|---|---| +| `bin/contentful.js` | Entry point — requires compiled `dist/lib/cli.js` | +| `lib/cli.ts` | Yargs CLI setup, top-level command registration | +| `lib/cmds/` | Top-level commands (`login`, `logout`, `init`, `space`, `merge`, `sync`, `organization`, etc.) | +| `lib/cmds/_cmds/` | Subcommands (e.g., `space_cmds/create.ts`, `organization_cmds/sec-check.ts`) | +| `lib/utils/` | Shared utilities — API clients, error handling, proxy, pagination, merge logic | +| `lib/utils/merge/` | Merge-specific utilities (changeset rendering, content type helpers) | +| `lib/core/events/` | Internal event system for space creation and logging | +| `lib/config.js` | Command authorization lists (no-auth-needed, no-space-id-needed) | +| `lib/context.js` | Context resolution — reads `.contentfulrc.json`, resolves active space, environment, token | +| `docs/` | Per-command documentation with usage examples | +| `test/unit/` | Unit tests (Jest) | +| `test/integration/` | Integration tests with talkback proxy recordings | +| `test/e2e/` | End-to-end tests against standalone binaries | +| `dist/` | Compiled output (TypeScript → CommonJS) | +| `build/` | Standalone binary output (`@yao-pkg/pkg`) | + +## Data Flow + +1. **Authentication**: `contentful login` opens the OAuth page in a browser, user copies CMA token back, token is stored in `~/.contentfulrc.json`. +2. **Context resolution**: Before each command, the CLI reads `.contentfulrc.json` for active space ID, environment ID, host, and management token. +3. **Command execution**: Yargs dispatches to the appropriate command handler in `lib/cmds/`. Commands use `contentful-management.js` to make API calls. +4. **Delegation**: Import, export, and migration commands delegate to their respective npm packages, passing configuration and the CMA client. +5. **Output**: Results are rendered to the terminal via `chalk`, `cli-table3`, `boxen`, and `listr` (task runner UI). + +## Domain Concepts + +The CLI operates on Contentful's content model. Key entities and their relationships: + +| Concept | Description | CLI Commands | +|---|---|---| +| **Space** | Top-level container for content | `space create`, `space list`, `space use` | +| **Environment** | Isolated branch of a space's content (e.g., `master`, `staging`) | `space environment create/list` | +| **Content Type** | Schema definition for entries (fields, validations, appearance) | `content-type list`, `content-type get` | +| **Entry** | An instance of a content type | Managed via import/export | +| **Asset** | A media file with metadata | Managed via import/export | +| **Extension** | UI extension for the Contentful web app | `extension create/update/delete` | +| **Migration** | Scripted schema change applied to an environment | `space migration` | + +The primary lifecycle is: **Space → Environment → Content Type → Entry/Asset**. Migrations modify content types within an environment. Import/export operates on entire spaces or environments. + + +## Key Dependencies + +| Dependency | Why it's here | +|---|---| +| `yargs` (~13.3.2) | CLI framework — command parsing, help generation, argument validation | +| `contentful-management` (^12.2.0) | CMA.js SDK — all API interactions ([ADR: CMA v12 migration](./docs/ADRs/2026-04-22-cma-v12-migration.md)) | +| `contentful-migration` (^5.0.0) | Migration script execution engine | +| `contentful-import` / `contentful-export` | Space import/export functionality | +| `contentful-batch-libs` (^11.0.0) | Shared batch processing utilities for import/export | +| `inquirer` (^8.2.7) | Interactive prompts (space selection, login, init) | +| `listr` | Task runner UI for multi-step operations | +| `chalk` | Terminal styling | +| `@yao-pkg/pkg` (dev) | Standalone binary compilation ([ADR: yao-pkg migration](./docs/ADRs/2026-04-10-yao-pkg-standalone-binaries.md)) | + +## Configuration + +| Variable / Flag | Purpose | Default | +|---|---|---| +| `~/.contentfulrc.json` | Global config — management token, active space/env, host | Created on `login` | +| `.contentfulrc.json` (local) | Per-project config — overrides global settings | None | +| `--management-token` / `--mt` | Override stored token for a single command | From config | +| `--space-id` | Override active space | From config | +| `--environment-id` | Override active environment | `master` | +| `--host` | CMA host (EU: `api.eu.contentful.com`) | `api.contentful.com` | +| `--proxy` | HTTP proxy in `user:auth@host:port` format | None | +| `http_proxy` / `https_proxy` | Environment variable proxy config | None | + +## Integration Points + +### Upstream (this repo consumes) + +- **Contentful Management API** — all CRUD operations on spaces, environments, content types, entries, assets, extensions, organizations +- **Contentful OAuth Page** (`contentful-cli-oauth-page` repo) — browser-based token acquisition during `contentful login` + +### Downstream (consumes this repo) + +- **Developers** — installed globally via `npm install -g contentful-cli` or `npx contentful-cli` +- **CI/CD pipelines** — migration execution, import/export, environment management in automated workflows +- **Standalone binary users** — macOS/Linux/Windows binaries attached to GitHub releases + +## Operational Knowledge + +This is a CLI tool, not a running service. The operational model differs from a typical backend. + +### Deployment + +Releases are automated via semantic-release on merge to `main` (or `beta` for pre-releases). The CI pipeline builds standalone binaries and publishes to npm + GitHub Releases. There is no staged rollout or canary process. + +**Rollback:** Publish a new patch version. npm versions can be deprecated via `npm deprecate`. Standalone binaries attached to GitHub releases cannot be recalled. + +### Failure Modes + +- **CMA API unavailable:** All commands that make API calls fail with HTTP errors propagated to stderr. No retry logic or circuit breaking. +- **OAuth page unavailable:** `contentful login` fails — users must manually set a token via `--management-token`. +- **Satellite package errors:** Import/export/migration failures surface the satellite package's error messages directly. + +### Monitoring + +There are no dashboards, alerts, or telemetry. Diagnosing issues with a release: check [GitHub Issues](https://github.com/contentful/contentful-cli/issues) for user reports, npm download stats for adoption anomalies, and the team's internal Slack channel for reports. + +### Incident Playbook + +No formal incident playbook exists for the CLI. Typical response is to triage GitHub issues and publish a patch release if a regression is confirmed. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9c426c418..ac12e1180 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,13 +6,15 @@ It also explains what to do in case you want to setup the project locally and ru # Setup -Run `npm install` or `yarn` to install all necessary dependencies. When running `npm install` or `yarn` locally, `dist` is not compiled. +Run `npm install` to install all dependencies. The `.npmrc` sets `ignore-scripts=true` for supply chain security, so after install run `npx allow-scripts` to allow trusted packages to execute their post-install scripts. (CI uses `npm ci` for reproducibility.) All necessary dependencies are installed under `node_modules` and any necessary tools can be accessed via npm scripts. There is no need to install anything globally. +You must compile TypeScript before running the CLI: `npm run tsc` (output goes to `dist/`). + # Code style -This project uses [standard](https://github.com/feross/standard). Install a relevant editor plugin if you'd like. +This project uses ESLint (`.eslintrc.js`) and Prettier (`.prettierrc`). Pre-commit hooks enforce formatting on changed `.js` files via `lint-staged`. Everywhere where it isn't applicable, follow a style similar to the existing code. @@ -56,23 +58,17 @@ See [jest](https://jestjs.io/) documentation for more details about running test ## Integration tests -To run integration tests locally, [talkback](https://github.com/ijpiantanida/talkback) is used as a proxy to record and playback http requests - -1. Prepare build in prep for integration tests -```sh -npm run build:standalone -``` +To run integration tests locally, [talkback](https://github.com/ijpiantanida/talkback) is used as a proxy to record and playback http requests. -1. In another terminal shell run your preferred tests ```sh -# Ensure environment variables are set to for the Ecosystem Integration Test Org (`Contentful - Ecosystem (for integration test org)` in password vault) +# Ensure environment variables are set for the Ecosystem Integration Test Org export CONTENTFUL_INTEGRATION_TEST_CMA_TOKEN='' export CLI_E2E_ORG_ID='' -# Run all integration tests +# Run all integration tests (starts talkback proxy automatically via concurrently) npm run test:integration -# Or run specific tests +# Or run specific tests (start proxy separately first) npm run talkback-proxy npx jest test/integration/cmds/space/* --watch ``` @@ -87,10 +83,132 @@ Tip: run tests without recordings to update the snapshots. npx jest test/integration/cmds/ --updateSnapshot ``` -This project has unit and integration tests. Both of these run on both Node.js and Browser environments. - -Both of these test environments are setup to deal with Babel and code transpiling, so there's no need to worry about that +This project has unit and integration tests. Both run in Node.js. Jest uses Babel for TypeScript transpilation during tests. # Other tasks - `npm run build:standalone` build standalone binary version + +--- + + + + +# Additional Context + +## Prerequisites + +| Tool | Version | Notes | +|---|---|---| +| Node.js | 24 (see `.nvmrc`); >=22 supported | Use `nvm use` to switch automatically | +| npm | 10+ | Ships with Node.js 24 | + +## Build & Type-Check + +```bash +npm run tsc # source: package.json → scripts.tsc +npx tsc --noEmit # source: tsconfig.json (type-check without emitting) +npm run tsc:watch # source: package.json → scripts.tsc:watch +``` + +> **Note on linting:** ESLint is configured (`.eslintrc.js`) but linting is not enforced in CI — the lint job is currently commented out in the GitHub Actions workflow. Pre-commit hooks run `prettier` and `lint-staged` on changed `.js` files only. + +> **This is a CLI, not a service.** There is no `npm start` or long-running process. To "run it locally," build with `npm run tsc` and then invoke commands via `node bin/contentful.js ` or via `npm link`. + +## Development Workflow + +To avoid your development version colliding with a globally installed `contentful` binary, change the command name in `package.json`: + +```diff + "bin": { +- "contentful": "bin/contentful.js" ++ "ctfl": "bin/contentful.js" + } +``` + +Then link your local version: + +```bash +npm link +``` + +You can also run commands directly from source without linking: + +```bash +node bin/contentful.js space list +node bin/contentful.js login +node bin/contentful.js content-type list --space-id +``` + +## E2E Tests + +E2E tests run against standalone binaries on macOS and Linux: + +```bash +npm run build:standalone # source: package.json → scripts.build:standalone +npm run test:e2e # source: package.json → scripts.test:e2e +``` + +## Breaking Changes + +Include `BREAKING CHANGE:` in the commit footer. The `release` config in `package.json` maps `{ "breaking": true }` to a major release, and `{ "type": "build", "scope": "deps" }` to a patch release. + +## Branch Strategy + +- `main` — production releases, auto-published via semantic-release +- `beta` — pre-release channel (`npm install contentful-cli@beta`) +- Feature branches — any naming convention, CI runs on all branches + +## Release Process + +Releases are fully automated via [semantic-release](https://github.com/semantic-release/semantic-release) on GitHub Actions: + +1. Merge to `main` (or `beta` for pre-releases) +2. CI runs: build → check (unit + integration tests) → e2e tests +3. semantic-release analyzes commits, determines version bump +4. Publishes to npm, creates GitHub release with standalone binaries (macOS, Linux, Windows `.zip`) + +Standalone binaries are built with `@yao-pkg/pkg` targeting Node 22 for macOS x64, Linux x64, and Windows x64. + +## Adding a New Command + +Use an existing simple command as a reference pattern. For a top-level command, see `lib/cmds/logout.js`. For a subcommand, see `lib/cmds/space_cmds/list.ts`. + +Every command file exports a yargs command object with `command`, `desc`, `builder`, and `handler`. Subcommands go in `lib/cmds/_cmds/` and are auto-discovered via yargs `commandDir`. + +If the command doesn't require authentication, add it to the `noAuthNeeded` array in `lib/config.js`. If it doesn't require a space ID, add it to `noSpaceIdNeeded`. + +## Pull Requests + +- All CI checks must pass (build, unit tests, integration tests, e2e tests) +- Squash merge to `main` is the standard workflow +- Dependabot PRs are auto-approved and merge-requested via workflow + +## CI/CD + +| Job | Trigger | What it does | +|---|---|---| +| Build | All pushes and PRs | Compiles TypeScript, builds standalone binaries, caches `dist/` and `build/` | +| Check | After build | Runs unit tests (with coverage) and integration tests (with talkback proxy) | +| E2E Tests | After build + check | Runs e2e tests against standalone binaries on ubuntu and macOS | +| Release | Push to `main` or `beta` | semantic-release: version bump, npm publish, GitHub release with binaries | +| CodeQL | Scheduled + PRs | Static analysis for security vulnerabilities | +| Dependabot Approve | Dependabot PRs | Auto-approves and requests merge for dependency updates | + +## File-Level Guidance + +| Path | Notes | +|---|---| +| `lib/context.js` | Resolves auth token, space ID, and environment from `.contentfulrc.json` — bugs here break every authenticated command | +| `lib/config.js` | Controls which commands skip auth/space-id checks — wrong entries cause silent auth bypass or false "not logged in" errors | +| `lib/cmds/login.ts` | OAuth token flow — security-sensitive, handles token storage | +| `lib/utils/proxy.js` | HTTP proxy setup — incorrect changes can silently break all API calls for proxy users | +| `lib/cli.ts` | Top-level yargs setup and command registration — changes here affect every command | +| `dist/` | Generated output — do not hand-edit, rebuild with `npm run tsc` | +| `build/` | Standalone binary output — do not hand-edit, rebuild with `npm run build:standalone` | + +## Code Style & Conventions + +- **Formatting:** Prettier (`.prettierrc`) — enforced via pre-commit hook on `.js` files +- **Linting:** ESLint (`.eslintrc.js`) — configured but not enforced in CI (lint job commented out) +- **TypeScript:** `tsconfig.json` — `strict: true`, `allowJs: true`, target `es2016`, CommonJS output diff --git a/README.md b/README.md index c011d7558..1d4345a33 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ > [Contentful's](https://www.contentful.com) command line interface tool. Use Contentful features straight from your CLI. [![npm](https://img.shields.io/npm/v/contentful-cli.svg)](https://www.npmjs.com/package/contentful-cli) -[![Contentful](https://circleci.com/gh/contentful/contentful-cli.svg?style=shield)](https://circleci.com/gh/contentful/contentful-cli) +[![Build](https://github.com/contentful/contentful-cli/actions/workflows/main.yaml/badge.svg)](https://github.com/contentful/contentful-cli/actions/workflows/main.yaml) [Contentful](https://www.contentful.com) provides a content infrastructure for digital teams to power content in websites, apps, and devices. Unlike a CMS, Contentful was built to integrate with the modern software stack. It offers a central hub for structured content, powerful management and delivery APIs, and a customizable web app that enable developers and content creators to ship digital products faster. @@ -117,6 +117,11 @@ If you have other problems with Contentful not related to this library, you can See [CONTRIBUTING.md](CONTRIBUTING.md) +## For AI Agents + + +If you are an AI coding agent working in this repository, read [AGENTS.md](./AGENTS.md) first. It tells you where to find architectural context, development setup, decision records, and repo-specific rules. + ## :scroll: License MIT diff --git a/docs/ADRs/2017-08-31-yargs-cli-framework.md b/docs/ADRs/2017-08-31-yargs-cli-framework.md new file mode 100644 index 000000000..337877837 --- /dev/null +++ b/docs/ADRs/2017-08-31-yargs-cli-framework.md @@ -0,0 +1,24 @@ +# Yargs as CLI Framework + +## Status + +Accepted + +## Context + +When the CLI was created in 2017 (`feat(init): A new beginning`), a framework was needed to handle command parsing, argument validation, help generation, and hierarchical subcommand routing (e.g., `contentful space environment create`). + +The major options at the time were yargs, commander, and oclif. Yargs was widely adopted, lightweight, and supported the nested command pattern (`commandDir`) that maps well to the CLI's file-based command structure. + +## Decision + +Use yargs (~13.3.2) as the CLI framework. Commands are organized as files in `lib/cmds/`, with subcommands in `lib/cmds/_cmds/` directories. Yargs auto-discovers these via its `commandDir` pattern. + +The version is pinned to the 13.x range. Upgrading has not been prioritized since the current version is stable and meets all requirements. + +## Consequences + +- The `lib/cmds/` directory structure directly maps to the CLI's command hierarchy, making it easy to find and add commands +- Yargs 13.x is significantly behind current versions (17.x) — a major version bump would require updating all command definitions +- The pinned version means some newer yargs features (middleware improvements, async completions) are unavailable +- Interactive prompts are handled separately via `inquirer`, not yargs diff --git a/docs/ADRs/2018-08-30-talkback-integration-tests.md b/docs/ADRs/2018-08-30-talkback-integration-tests.md new file mode 100644 index 000000000..276fba635 --- /dev/null +++ b/docs/ADRs/2018-08-30-talkback-integration-tests.md @@ -0,0 +1,30 @@ +# Talkback Proxy for Integration Tests + +## Status + +Accepted + +## Context + +Integration tests need to exercise real CLI commands against the Contentful API, but hitting the API directly in CI is slow, flaky, and requires credentials. The tests needed a way to record and replay HTTP interactions deterministically. + +Talkback was introduced in August 2018 (`test(integration): Add recordings for integration tests`). + +## Decision + +Use [talkback](https://github.com/ijpiantanida/talkback) as an HTTP proxy that records API responses as "tapes" and replays them on subsequent runs. The proxy runs alongside tests via `concurrently`: + +```bash +npm run test:integration # starts talkback-proxy + jest concurrently +``` + +The proxy is configured in `test/proxy.js` and recorded interactions are stored alongside the test files. + +## Consequences + +- Integration tests are fast and deterministic in CI — no real API calls needed +- New integration tests may need fresh recordings when API responses change +- Updating snapshots requires running tests without recordings first, then re-recording +- Unit tests still depend on integration test environment variables being set (a known coupling) +- E2E tests (separate from integration tests) run against real standalone binaries and may hit the API +- Recorded tapes may contain non-credential identifiers from API responses (space IDs, environment IDs, user metadata) — recordings should only be made against the dedicated integration test org, never against orgs with real customer data diff --git a/docs/ADRs/2019-02-21-semantic-release.md b/docs/ADRs/2019-02-21-semantic-release.md new file mode 100644 index 000000000..7859033e2 --- /dev/null +++ b/docs/ADRs/2019-02-21-semantic-release.md @@ -0,0 +1,29 @@ +# Semantic Release for Automated Versioning and Publishing + +## Status + +Accepted + +## Context + +Manual versioning and publishing is error-prone and creates bottlenecks. The CLI needed a way to automatically determine version bumps from commit messages and publish to npm without human intervention. + +Semantic-release was adopted in February 2019 (PR #122, `config(ci): add semantic release`). + +## Decision + +Adopt [semantic-release](https://github.com/semantic-release/semantic-release) with the Angular commit convention. Configuration in `package.json`: + +- **Release branches:** `main` (stable) and `beta` (pre-release channel) +- **Plugins:** commit-analyzer → release-notes-generator → npm → github (with binary assets) +- **Custom rules:** `{ "breaking": true, "release": "major" }` and `{ "type": "build", "scope": "deps", "release": "patch" }` + +The version in `package.json` is `0.0.0-determined-by-semantic-release` — the actual version is determined at release time. + +## Consequences + +- Every merge to `main` that includes `feat:` or `fix:` commits triggers a release automatically +- Dependency bumps (`build(deps):`) trigger patch releases +- Breaking changes require `BREAKING CHANGE:` in the commit footer to trigger major releases +- Pre-release versions are available on the `beta` npm channel +- Contributors must follow the commit convention or releases won't be generated correctly diff --git a/docs/ADRs/2022-07-29-typescript-adoption.md b/docs/ADRs/2022-07-29-typescript-adoption.md new file mode 100644 index 000000000..36cf3e670 --- /dev/null +++ b/docs/ADRs/2022-07-29-typescript-adoption.md @@ -0,0 +1,30 @@ +# TypeScript Adoption + +## Status + +Accepted + +## Context + +The CLI was originally written entirely in JavaScript. As the codebase grew and more contributors worked on it, the lack of type safety led to regressions and made refactoring risky. The broader Contentful SDK ecosystem was also moving toward TypeScript. + +The migration was incremental — new files were written in TypeScript while existing JavaScript files were left in place, allowing both to coexist via `allowJs: true` in `tsconfig.json`. + +## Decision + +Adopt TypeScript with an incremental migration strategy (PR #1570, `feat: init typescript`). Key choices: + +- **Target:** ES2016 (aligned with Node.js LTS support at the time) +- **Module system:** CommonJS (`"module": "commonjs"`) to avoid breaking the existing `require()`-based codebase +- **`allowJs: true`:** Enables gradual migration — `.js` and `.ts` files coexist +- **`strict: true`:** Enforced from the start for new TypeScript files +- **Compilation:** `tsc` compiles `lib/` → `dist/`, `bin/contentful.js` requires `dist/lib/cli.js` + +Babel + `@babel/preset-typescript` is used in the Jest test configuration for transpilation during testing. + +## Consequences + +- New commands and utilities are written in TypeScript (e.g., `login.ts`, `merge.ts`, `organization.ts`, `space_cmds/create.ts`) +- Many older files remain as `.js` (e.g., `config.js`, `context.js`, most extension commands) +- The mixed codebase means contributors must be comfortable with both `.js` and `.ts` +- The build step (`npm run tsc`) is required before running the CLI from source diff --git a/docs/ADRs/2025-11-14-gha-publishing.md b/docs/ADRs/2025-11-14-gha-publishing.md new file mode 100644 index 000000000..93c20b381 --- /dev/null +++ b/docs/ADRs/2025-11-14-gha-publishing.md @@ -0,0 +1,31 @@ +# Migration to GitHub Actions for CI/CD and Publishing + +## Status + +Accepted + +## Context + +The CLI historically used CircleCI for CI/CD (visible in the README badge). As Contentful standardized on GitHub Actions across the organization, the CLI needed to migrate its build, test, and release pipelines. + +This migration was tracked under DX-535. + +## Decision + +Migrate all CI/CD to GitHub Actions (PR #3160, `chore: migrate to gha publishing`). The pipeline was structured as reusable workflow calls: + +- `main.yaml` — orchestrator: triggers build → check → e2e → release +- `build.yaml` — compiles TypeScript, builds standalone binaries, caches artifacts +- `check.yaml` — unit tests + integration tests (with talkback proxy) +- `test-e2e.yaml` — runs e2e tests against standalone binaries on ubuntu and macOS (matrix, `max-parallel: 1`) +- `release.yaml` — semantic-release with Vault-sourced credentials (`contentful-automation[bot]`) + +Secrets are retrieved from HashiCorp Vault at runtime rather than stored as GitHub Secrets, following Contentful's security practices. + +## Consequences + +- CircleCI is fully decommissioned for this repo (badge in README is now stale) +- Build artifacts are shared between jobs via `actions/cache` with a run-specific key +- Release requires `contents: write` and `id-token: write` permissions for Vault JWT auth +- E2E tests run on both ubuntu and macOS to validate standalone binaries across platforms +- Linting and formatting checks are currently commented out in the check workflow diff --git a/docs/ADRs/2025-12-02-lavamoat-allow-scripts.md b/docs/ADRs/2025-12-02-lavamoat-allow-scripts.md new file mode 100644 index 000000000..1e7080cef --- /dev/null +++ b/docs/ADRs/2025-12-02-lavamoat-allow-scripts.md @@ -0,0 +1,26 @@ +# LavaMoat allow-scripts for Supply Chain Security + +## Status + +Accepted + +## Context + +npm packages can run arbitrary scripts during installation (`postinstall`, `preinstall`, etc.), creating a supply chain attack vector. The `.npmrc` setting `ignore-scripts=true` blocks all install scripts, but some legitimate packages (like `husky` for git hooks and `@yao-pkg/pkg`'s esbuild dependency) require their install scripts to function. + +## Decision + +Use `@lavamoat/allow-scripts` to selectively permit install scripts from trusted packages while blocking all others. The allowlist is configured in `package.json` under `lavamoat.allowScripts`: + +- `$root$` — the project's own scripts +- `husky` — git hook setup +- `inquirer-select-directory>inquirer>external-editor>spawn-sync` — required for interactive directory selection +- `@yao-pkg/pkg>esbuild` — binary compilation tooling + +After `npm ci` (which honors `ignore-scripts=true`), `npx allow-scripts` is run to execute only the approved scripts. + +## Consequences + +- All CI workflows and local setup must run `npx allow-scripts` after `npm ci` +- New dependencies that require install scripts must be explicitly added to the allowlist +- Reduces the risk of supply chain attacks via malicious install scripts diff --git a/docs/ADRs/2026-04-10-yao-pkg-standalone-binaries.md b/docs/ADRs/2026-04-10-yao-pkg-standalone-binaries.md new file mode 100644 index 000000000..25a548378 --- /dev/null +++ b/docs/ADRs/2026-04-10-yao-pkg-standalone-binaries.md @@ -0,0 +1,29 @@ +# Migration from pkg to @yao-pkg/pkg for Standalone Binaries + +## Status + +Accepted + +## Context + +The CLI is distributed both as an npm package and as standalone binaries for macOS, Linux, and Windows. These binaries are built by bundling the Node.js runtime with the compiled CLI code. + +The original `pkg` package by Vercel was deprecated and unmaintained. `@yao-pkg/pkg` is a community fork that continues maintenance and supports newer Node.js versions (the CLI targets Node 22). + +## Decision + +Migrate from `pkg` to `@yao-pkg/pkg` (commit `feat(ci) migrate pkg to yao-pkg to create linux/window/mac executables`, 2026-04-10). + +Build targets in `package.json`: +- `node22-macos-x64` +- `node22-linux-x64` +- `node22-win-x64` + +Assets bundled: `figlet` fonts and `axios` CJS bundle (required for proper bundling). + +## Consequences + +- Standalone binaries can target Node 22, matching the runtime requirement +- Binary artifacts are attached to GitHub releases as `.zip` files (macOS, Linux, Windows) +- E2E tests validate the binaries work correctly on ubuntu and macOS runners +- ARM/Apple Silicon macOS binaries are not built — macOS x64 runs via Rosetta 2 diff --git a/docs/ADRs/2026-04-22-cma-v12-migration.md b/docs/ADRs/2026-04-22-cma-v12-migration.md new file mode 100644 index 000000000..0cddeb754 --- /dev/null +++ b/docs/ADRs/2026-04-22-cma-v12-migration.md @@ -0,0 +1,31 @@ +# CMA.js v12 Migration and Node.js 22+ Requirement + +## Status + +Accepted + +## Context + +`contentful-management.js` (CMA.js) v12 was a major release that modernized the SDK. The CLI needed to upgrade to stay compatible with the latest API features and to align with the broader SDK ecosystem's migration tracked under DX-780 and DX-689. + +CMA.js v12 introduced breaking changes including removal of deprecated methods, changes to `createPlainClient()`, and updated type exports (e.g., `CursorPaginatedCollectionProp` moved). These changes cascaded through the CLI's satellite packages — `contentful-migration` (v5), `contentful-import` (v10), `contentful-export` (v8), and `contentful-batch-libs` (v11) all needed coordinated major bumps. + +## Decision + +Ship the CMA v12 migration as a single breaking change (PR #3193, `feat!: update to CMA.js v12 and drop support for Node < 22`). This: + +- Bumped `contentful-management` to ^12.2.0 +- Updated `contentful-migration` to v5.0.0, `contentful-import` to v10.0.0, `contentful-export` to v8.0.0, `contentful-batch-libs` to v11.0.0 +- Dropped Node.js support below v22 (`engines.node: ">=22"`) +- Migrated `createPlainClient()` to explicitly use the "legacy" deprecated CMA where needed +- Updated deep imports that moved in v12 (e.g., `CursorPaginatedCollectionProp`) +- Added `{ "breaking": true, "release": "major" }` to semantic-release config to correctly trigger major version bumps + +This resulted in `contentful-cli` v4.0.0. + +## Consequences + +- Users on Node.js < 22 must upgrade before using CLI v4+ +- All satellite packages are now on their latest majors, aligned with CMA.js v12 +- CI builds and standalone binaries target Node 22/24 +- Some `createPlainClient()` call sites use the "legacy" mode as a transitional measure — these should be migrated to the new API surface over time diff --git a/docs/ADRs/README.md b/docs/ADRs/README.md new file mode 100644 index 000000000..e141578e4 --- /dev/null +++ b/docs/ADRs/README.md @@ -0,0 +1,12 @@ +# Architecture Decision Records + +| ADR | Date | Status | Title | +|---|---|---|---| +| [001](./2017-08-31-yargs-cli-framework.md) | 2017-08-31 | Accepted | Yargs as CLI Framework | +| [002](./2018-08-30-talkback-integration-tests.md) | 2018-08-30 | Accepted | Talkback Proxy for Integration Tests | +| [003](./2019-02-21-semantic-release.md) | 2019-02-21 | Accepted | Semantic Release for Automated Versioning and Publishing | +| [004](./2022-07-29-typescript-adoption.md) | 2022-07-29 | Accepted | TypeScript Adoption | +| [005](./2025-11-14-gha-publishing.md) | 2025-11-14 | Accepted | Migration to GitHub Actions for CI/CD and Publishing | +| [006](./2025-12-02-lavamoat-allow-scripts.md) | 2025-12-02 | Accepted | LavaMoat allow-scripts for Supply Chain Security | +| [007](./2026-04-10-yao-pkg-standalone-binaries.md) | 2026-04-10 | Accepted | Migration from pkg to @yao-pkg/pkg for Standalone Binaries | +| [008](./2026-04-22-cma-v12-migration.md) | 2026-04-22 | Accepted | CMA.js v12 Migration and Node.js 22+ Requirement | diff --git a/docs/specs/.gitkeep b/docs/specs/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/specs/README.md b/docs/specs/README.md new file mode 100644 index 000000000..acffee262 --- /dev/null +++ b/docs/specs/README.md @@ -0,0 +1,3 @@ +# Specs +Implementation-level specs for active work in this repo. Format: `YYYY-MM-.md`. +Specs here represent current intent — archive or remove when work ships.