Skip to content

Commit 39841fd

Browse files
chore(cli-generator): add cli-sdk sync pipeline and documentation (#16149)
* chore(cli-generator): add cli-sdk sync pipeline and documentation - Add generators/cli/scripts/sync-sdk.sh: deterministic projection of cli-sdk's workspace Cargo.toml into the vendored single-package manifest, rsync of source files under SDK_IGNORE rules, Cargo.lock regeneration, and provenance marker (.synced-from). - Add .github/workflows/sync-cli-sdk.yml: daily cron + workflow_dispatch that checks out cli-sdk main HEAD, runs the sync script, opens an auto-merging PR if there are changes, and relies on seed tests as the trust boundary. - Update generators/cli/CLAUDE.md with sync documentation covering projection rules, manual sync instructions, and the must-rebuild list. Closes FER-10795 * fix: address Hex Security findings — remove auto-merge/self-approval, pin actions to SHA - Remove 'Enable auto-merge' and 'Approve PR' steps to require human review before externally-sourced code lands in main (high severity) - Pin peter-evans/create-pull-request to commit SHA instead of mutable tag (medium severity) - Remove unnecessary pull-requests:write permission - Update CLAUDE.md to reflect human review requirement * fix: remove dead rsync exclude for src/bin/strip_schema.rs The exclude pattern used the wrong relative path (rsync paths are relative to the transfer root, which is $CLI_SDK_DIR/src/). The file is also required by the projected Cargo.toml [[bin]] entry for strip-schema, so excluding it would break cargo build. * fix: remove timestamp from .synced-from to prevent spurious daily PRs The timestamp changed on every run, causing git diff to always report changes even when cli-sdk SHA is unchanged. Only the SHA is needed for provenance — git commit timestamps provide the sync date. * chore: add cargo build --locked verification step to sync workflow Validates the projected SDK compiles before opening a PR, so a broken sync never reaches reviewers. * fix: use extract_section for workspace version extraction grep -A2 was fragile — would fail if version wasn't within 2 lines of the [workspace.package] header. Also removes unused SYNC_DATE variable. --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent cf5b2d7 commit 39841fd

3 files changed

Lines changed: 347 additions & 0 deletions

File tree

.github/workflows/sync-cli-sdk.yml

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
name: Sync cli-sdk into generators/cli/sdk
2+
3+
on:
4+
schedule:
5+
# Daily at 06:00 UTC (after US-Pacific EOD, before EU morning)
6+
- cron: "0 6 * * *"
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: write
11+
12+
concurrency:
13+
group: sync-cli-sdk
14+
cancel-in-progress: true
15+
16+
env:
17+
DO_NOT_TRACK: "1"
18+
19+
jobs:
20+
sync:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- name: Checkout fern
24+
uses: actions/checkout@v6
25+
with:
26+
ref: main
27+
28+
- name: Checkout cli-sdk at main HEAD
29+
uses: actions/checkout@v6
30+
with:
31+
repository: fern-api/cli-sdk
32+
ref: main
33+
path: _cli-sdk
34+
token: ${{ secrets.FERN_SUPPORT_GH_ACTIONS_PAT }}
35+
36+
- name: Install Rust toolchain
37+
uses: dtolnay/rust-toolchain@stable
38+
39+
- name: Run sync script
40+
run: bash generators/cli/scripts/sync-sdk.sh _cli-sdk
41+
42+
- name: Verify projected SDK builds
43+
run: cargo build --locked --all-features --tests
44+
working-directory: generators/cli/sdk
45+
46+
- name: Check for changes
47+
id: diff
48+
run: |
49+
git add -A generators/cli/sdk/
50+
if git diff --cached --quiet; then
51+
echo "changed=false" >> "$GITHUB_OUTPUT"
52+
else
53+
echo "changed=true" >> "$GITHUB_OUTPUT"
54+
fi
55+
56+
- name: Read provenance
57+
if: steps.diff.outputs.changed == 'true'
58+
id: provenance
59+
run: |
60+
SHA="$(head -1 generators/cli/sdk/.synced-from | sed 's/cli-sdk@//')"
61+
SHORT="${SHA:0:7}"
62+
echo "sha=$SHA" >> "$GITHUB_OUTPUT"
63+
echo "short=$SHORT" >> "$GITHUB_OUTPUT"
64+
65+
- name: Create pull request
66+
if: steps.diff.outputs.changed == 'true'
67+
id: cpr
68+
uses: peter-evans/create-pull-request@98357b18bf14b5342f975ff684046ec3b2a07725 # v8
69+
with:
70+
token: ${{ secrets.FERN_SUPPORT_GH_ACTIONS_PAT }}
71+
commit-message: "chore(cli-generator): sync cli-sdk@${{ steps.provenance.outputs.short }}"
72+
title: "chore(cli-generator): sync cli-sdk@${{ steps.provenance.outputs.short }}"
73+
body: |
74+
Automated sync of [fern-api/cli-sdk](https://github.com/fern-api/cli-sdk) `main` HEAD into `generators/cli/sdk/`.
75+
76+
**Source**: `cli-sdk@${{ steps.provenance.outputs.sha }}`
77+
78+
Generated by `.github/workflows/sync-cli-sdk.yml` via `generators/cli/scripts/sync-sdk.sh`.
79+
Seed tests (`pnpm seed test --generator cli`) are the trust boundary — if they pass, a human reviewer can merge.
80+
branch: chore/sync-cli-sdk
81+
delete-branch: true
82+
labels: |
83+
cli-generator
84+
automated

generators/cli/CLAUDE.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,64 @@ The image warms a cargo target cache against the committed
197197
`Cargo.lock`; mounted fixtures use `cargo build --locked` and would
198198
otherwise refuse to start when the dep tree drifts.
199199

200+
### Syncing the vendored SDK from cli-sdk
201+
202+
The SDK at [`./sdk/`](./sdk/) is a **vendored snapshot** of
203+
[`fern-api/cli-sdk`](https://github.com/fern-api/cli-sdk). A daily
204+
GitHub Actions workflow (`.github/workflows/sync-cli-sdk.yml`) pulls
205+
`cli-sdk` `main` HEAD into this directory, opens a PR for human review,
206+
and relies on seed tests + a human reviewer as the trust boundary.
207+
208+
**How the sync works:**
209+
210+
[`generators/cli/scripts/sync-sdk.sh`](scripts/sync-sdk.sh) takes a
211+
local cli-sdk checkout and:
212+
213+
1. **Rsyncs source files** (`src/`, `tests/`, `cli/openapi-fixture/`)
214+
under the same `SDK_IGNORE` rules as `build.mjs` — template-only
215+
files (smoke tests, demo binaries, `.github/`, `docs/`, etc.) are
216+
excluded.
217+
2. **Projects `Cargo.toml`** — the vendored manifest is a deterministic
218+
projection of cli-sdk's workspace manifest, **not** a copy. A naïve
219+
`cp` would re-introduce `[workspace]`, `version.workspace = true`,
220+
and ~35 `[[bin]]` entries. The projection:
221+
- **Drops** `[workspace]` + `[workspace.package]`
222+
- **Keeps only** the `openapi-fixture` and `strip-schema` `[[bin]]`
223+
entries
224+
- **Rewrites** `version.workspace = true` → literal
225+
`version = "<synced>"`
226+
- **Injects** the 3 Fern comment blocks that `patchCargoToml.ts`
227+
anchors on (`TEMPLATE_TOP_COMMENT`, `TEMPLATE_BIN_COMMENT`, the
228+
`strip-schema` "Internal tool…" comment) plus `readme = "README.md"`
229+
and `[package.metadata.dist] dist = false`
230+
- **Copies verbatim**: `[dependencies]`, `[features]`, `[lib]`,
231+
`[profile.dist]`, `[build-dependencies]`, `[dev-dependencies]`
232+
3. **Regenerates `Cargo.lock`** so `cargo build --locked` is honest.
233+
4. **Writes `.synced-from`** with `cli-sdk@<sha>` + timestamp for
234+
provenance tracking.
235+
236+
**Manual sync** (when you can't wait for the daily cron):
237+
238+
```bash
239+
# From the fern repo root:
240+
git clone --depth 1 https://github.com/fern-api/cli-sdk.git /tmp/cli-sdk
241+
bash generators/cli/scripts/sync-sdk.sh /tmp/cli-sdk
242+
# Review the diff, then commit.
243+
```
244+
245+
**Must-rebuild list** (only when `Cargo.lock` changes):
246+
247+
```bash
248+
pnpm turbo run dist:cli --filter @fern-api/cli-generator
249+
docker build --no-cache -f docker/seed/Dockerfile.cli -t fernapi/cli-seed:latest .
250+
pnpm turbo run dist:cli --filter @fern-api/seed-cli
251+
```
252+
253+
**Key invariant**: every `patchCargoToml.ts` anchor must be present in
254+
the projected `Cargo.toml`. If you rename a comment block in the
255+
template, update `sync-sdk.sh`'s projection accordingly — the
256+
`patchCargoToml.test.ts` will catch any mismatch at test time.
257+
200258
## Conventions
201259

202260
- **No TOML parser**: `patchCargoToml` uses literal string replacement

generators/cli/scripts/sync-sdk.sh

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
#!/usr/bin/env bash
2+
# sync-sdk.sh — deterministic sync of fern-api/cli-sdk into generators/cli/sdk/
3+
#
4+
# Usage:
5+
# generators/cli/scripts/sync-sdk.sh <path-to-cli-sdk-checkout>
6+
#
7+
# The script expects a local checkout of cli-sdk (at the desired ref) as its
8+
# sole argument. It projects the cli-sdk workspace Cargo.toml into the
9+
# single-package vendored Cargo.toml, rsyncs source files, regenerates
10+
# Cargo.lock, and writes a provenance marker.
11+
#
12+
# Called by .github/workflows/sync-cli-sdk.yml (daily) and can be run
13+
# manually for ad-hoc syncs.
14+
set -euo pipefail
15+
16+
if [[ $# -ne 1 ]]; then
17+
echo "Usage: $0 <path-to-cli-sdk-checkout>" >&2
18+
exit 1
19+
fi
20+
21+
CLI_SDK_DIR="$(cd "$1" && pwd)"
22+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
23+
SDK_DIR="$(cd "$SCRIPT_DIR/../sdk" && pwd)"
24+
25+
if [[ ! -f "$CLI_SDK_DIR/Cargo.toml" ]]; then
26+
echo "Error: $CLI_SDK_DIR/Cargo.toml not found" >&2
27+
exit 1
28+
fi
29+
30+
CLI_SDK_SHA="$(git -C "$CLI_SDK_DIR" rev-parse HEAD 2>/dev/null || echo "unknown")"
31+
CLI_SDK_SHORT="$(git -C "$CLI_SDK_DIR" rev-parse --short HEAD 2>/dev/null || echo "unknown")"
32+
33+
echo "==> Syncing cli-sdk@${CLI_SDK_SHORT} (${CLI_SDK_SHA}) into generators/cli/sdk/"
34+
35+
# ---------------------------------------------------------------------------
36+
# 1. Rsync source files with SDK_IGNORE rules (mirrors build.mjs SDK_IGNORE)
37+
# ---------------------------------------------------------------------------
38+
echo "--- Syncing src/, tests/, cli/openapi-fixture/ ..."
39+
40+
rsync -a --delete \
41+
--exclude='.DS_Store' \
42+
--exclude='target/' \
43+
--exclude='.gitignore' \
44+
--exclude='docs/' \
45+
--exclude='tests/overlay_fixture.rs' \
46+
--exclude='tests/fixtures/' \
47+
--exclude='cli/openapi-fixture/' \
48+
--exclude='.github/' \
49+
--exclude='build.rs' \
50+
--exclude='tests/common/' \
51+
--exclude='tests/auth_routing_wire.rs' \
52+
--exclude='tests/extension_surface_behavior.rs' \
53+
--exclude='tests/lib_api.rs' \
54+
--exclude='tests/tls_env_vars.rs' \
55+
--exclude='changes/' \
56+
"$CLI_SDK_DIR/src/" "$SDK_DIR/src/"
57+
58+
# Sync tests (only the ones not in SDK_IGNORE)
59+
mkdir -p "$SDK_DIR/tests"
60+
rsync -a --delete \
61+
--exclude='.DS_Store' \
62+
--exclude='overlay_fixture.rs' \
63+
--exclude='fixtures/' \
64+
--exclude='common/' \
65+
--exclude='auth_routing_wire.rs' \
66+
--exclude='extension_surface_behavior.rs' \
67+
--exclude='lib_api.rs' \
68+
--exclude='tls_env_vars.rs' \
69+
"$CLI_SDK_DIR/tests/" "$SDK_DIR/tests/"
70+
71+
# Sync cli/openapi-fixture/ (the dev fixture used by seed)
72+
mkdir -p "$SDK_DIR/cli/openapi-fixture"
73+
rsync -a --delete \
74+
"$CLI_SDK_DIR/cli/openapi-fixture/" "$SDK_DIR/cli/openapi-fixture/"
75+
76+
# ---------------------------------------------------------------------------
77+
# 2. Project Cargo.toml (not a naive copy — workspace → single-package)
78+
# ---------------------------------------------------------------------------
79+
echo "--- Projecting Cargo.toml ..."
80+
81+
# Helper: extract all lines between a TOML [section] header and the next header.
82+
extract_section() {
83+
local file="$1" section="$2"
84+
awk -v sect="$section" '
85+
BEGIN { found=0 }
86+
/^\[/ {
87+
if (found) exit
88+
# Match the section header exactly
89+
gsub(/^[[:space:]]+|[[:space:]]+$/, "")
90+
if ($0 == "[" sect "]") { found=1; next }
91+
}
92+
found { print }
93+
' "$file"
94+
}
95+
96+
# Extract the workspace version from cli-sdk
97+
WORKSPACE_VERSION="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "workspace.package" \
98+
| grep '^version' | sed 's/.*"\(.*\)".*/\1/')"
99+
100+
if [[ -z "$WORKSPACE_VERSION" ]]; then
101+
echo "Error: could not extract [workspace.package] version from cli-sdk" >&2
102+
exit 1
103+
fi
104+
105+
echo " version: $WORKSPACE_VERSION"
106+
107+
FEATURES_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "features")"
108+
DEPS_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "dependencies")"
109+
BUILD_DEPS_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "build-dependencies")"
110+
DEV_DEPS_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "dev-dependencies")"
111+
PROFILE_DIST_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "profile.dist")"
112+
METADATA_DIST_BODY="$(extract_section "$CLI_SDK_DIR/Cargo.toml" "package.metadata.dist")"
113+
114+
# Build the projected Cargo.toml with Fern comment blocks intact
115+
cat > "$SDK_DIR/Cargo.toml" << 'CARGO_HEADER'
116+
# `name`, `repository`, `homepage`, `authors`, and `keywords` are Fern's —
117+
# they identify the SDK template's source on crates.io. The fern-cli
118+
# generator does NOT rewrite this block when producing your CLI; only the
119+
# [[bin]] entry below is templated. If you want to publish *your* CLI as
120+
# its own crate on crates.io, edit this block to your org's metadata.
121+
# The [lib] name (`fern_cli_sdk`) is the import path every `use
122+
# fern_cli_sdk::...` site in src/ depends on — do NOT rename it.
123+
[package]
124+
name = "fern-cli-sdk"
125+
CARGO_HEADER
126+
127+
cat >> "$SDK_DIR/Cargo.toml" << EOF
128+
version = "$WORKSPACE_VERSION"
129+
edition = "2021"
130+
description = "CLI generator — dynamic command surface from OpenAPI and GraphQL schemas"
131+
license = "Apache-2.0"
132+
repository = "https://github.com/fern-api/cli-sdk"
133+
homepage = "https://github.com/fern-api/cli-sdk"
134+
readme = "README.md"
135+
authors = ["Fern <hey@buildwithfern.com>"]
136+
keywords = ["cli", "openapi", "graphql", "fern", "codegen"]
137+
categories = ["command-line-utilities", "web-programming"]
138+
139+
[lib]
140+
name = "fern_cli_sdk"
141+
path = "src/lib.rs"
142+
143+
# Rewritten by the fern-cli generator's \`patchCargoToml\` step — both the
144+
# \`name\` and \`path\` are replaced with the derived binary name so users
145+
# get \`cargo install\`-able binaries named after their API rather than
146+
# the template's literal "openapi-fixture".
147+
[[bin]]
148+
name = "openapi-fixture"
149+
path = "cli/openapi-fixture/main.rs"
150+
151+
# Internal tool used by the SDK template itself — not the user's CLI.
152+
[[bin]]
153+
name = "strip-schema"
154+
path = "src/bin/strip_schema.rs"
155+
156+
[features]
157+
$FEATURES_BODY
158+
159+
[dependencies]
160+
$DEPS_BODY
161+
162+
[package.metadata.dist]
163+
$METADATA_DIST_BODY
164+
[profile.dist]
165+
$PROFILE_DIST_BODY
166+
167+
[build-dependencies]
168+
$BUILD_DEPS_BODY
169+
170+
[dev-dependencies]
171+
$DEV_DEPS_BODY
172+
EOF
173+
174+
# ---------------------------------------------------------------------------
175+
# 3. Provenance marker
176+
# ---------------------------------------------------------------------------
177+
echo "--- Writing provenance marker ..."
178+
179+
cat > "$SDK_DIR/.synced-from" << EOF
180+
cli-sdk@${CLI_SDK_SHA}
181+
EOF
182+
183+
# ---------------------------------------------------------------------------
184+
# 4. Regenerate Cargo.lock
185+
# ---------------------------------------------------------------------------
186+
echo "--- Regenerating Cargo.lock ..."
187+
(cd "$SDK_DIR" && cargo generate-lockfile 2>&1) || {
188+
echo "Warning: cargo generate-lockfile failed; Cargo.lock may be stale" >&2
189+
}
190+
191+
# ---------------------------------------------------------------------------
192+
# 5. Summary
193+
# ---------------------------------------------------------------------------
194+
echo ""
195+
echo "==> Sync complete: cli-sdk@${CLI_SDK_SHORT} → generators/cli/sdk/"
196+
echo " version: $WORKSPACE_VERSION"
197+
echo " sha: $CLI_SDK_SHA"
198+
echo " date: $SYNC_DATE"
199+
echo ""
200+
echo "Changed files:"
201+
(cd "$SDK_DIR" && git diff --stat 2>/dev/null || true)
202+
(cd "$SDK_DIR" && git diff --stat --cached 2>/dev/null || true)
203+
echo ""
204+
echo "Untracked files:"
205+
(cd "$SDK_DIR" && git ls-files --others --exclude-standard 2>/dev/null || true)

0 commit comments

Comments
 (0)