Skip to content

feat: add extract-code-examples composite action - #1

Merged
gavinsharp merged 13 commits into
mainfrom
extract-code-examples
May 11, 2026
Merged

feat: add extract-code-examples composite action#1
gavinsharp merged 13 commits into
mainfrom
extract-code-examples

Conversation

@gavinsharp

@gavinsharp gavinsharp commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a composite GitHub Action at extract-code-examples/ that any Phenoml SDK repo (TS, Python, Java) can call to extract Fern-generated code examples into a code-examples.json manifest for the docs generator. The extraction script auto-detects the SDK language from .fern/metadata.json and dispatches to a TypeScript, Python, or Java parser.

Layout

extract-code-examples/        # one composite action per top-level dir
  action.yml
  index.ts                    # CLI + parsers
  package.json + bun.lock
  tests/
    extract-code-examples.test.ts
    fixtures/{typescript,python,java}/...
.github/workflows/
  test.yml                    # matrix-runs each action's test suite
README.md                     # usage docs

Each future action gets its own top-level directory. The test workflow uses a single-entry matrix so adding another is a one-line entry.

Design notes

  • Composite action rather than reusable workflow (workflow_call). The action runs inside the caller's existing job after their own actions/checkout, so no workflow_ref parsing, no double-checkout, and no implicit auto-commit. Callers decide whether/how to commit the manifest.
  • Self-locating: the action uses github.action_path to find its bundled script and package.json, so it works at any pinned ref.
  • Caller usage (from README.md):
    - uses: actions/checkout@v6
      with:
        ref: ${{ github.head_ref }}
    
    - uses: PhenoML/sdk-shared-actions/extract-code-examples@v1
    
    - name: Commit code-examples.json if changed
      run: |
        git add code-examples.json
        if ! git diff --cached --quiet -- code-examples.json; then
          git config user.name "github-actions[bot]"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git commit -m "chore: update code-examples.json"
          git push
        fi

The TS SDK currently has an inlined copy of this script (in PhenoML/phenoml-ts-sdk#139); once this lands and gets tagged, that PR can be reduced to the caller stub above.

Tests

38-test bun:test suite covering both unit-level helpers and end-to-end parsing of real SDK files (Summary client + wire tests for TS, authtoken auth raw_client + wire tests for Python, RawAuthClient + AuthtokenAuthWireTest for Java).

Several parser bugs caught and fixed during review now have regression coverage:

  • Python: def test_* boundary was matched on a trimmed line, so indented nested defs prematurely terminated body scanning.
  • Python: 60-line scan cap silently dropped long generated wire tests.
  • Java: brace-depth counting included braces inside string/char/comment/text-block literals — JSON fragments could corrupt method boundary detection.
  • Java: method-detection regex [^>]+ couldn't match nested generics like PhenomlClientHttpResponse<Optional<Foo>>.
  • Manifest builder: lowercased-concat chain-index keys could silently collapse different chains onto the same entry; collisions now warn + drop the ambiguous index entry rather than guess.

Test plan

  • bun run test passes locally (38 tests)
  • End-to-end CLI run against fixtures for all three languages
  • Tag a release (e.g. v1) once merged
  • Wire up the TS SDK's caller workflow against the tag and confirm code-examples.json is generated and committed
  • Repeat for Python and Java SDKs (once their wire-test infra lands)

🤖 Generated with Claude Code


Note

Medium Risk
Medium risk because this introduces a large new composite action and parsing logic that will run in SDK CI and write code-examples.json, so failures or mis-parsing could affect downstream docs/automation.

Overview
Adds new composite action extract-code-examples that runs a Bun/TypeScript CLI to parse Fern-generated TypeScript, Python, and Java SDKs and emit a code-examples.json manifest keyed by HTTP_METHOD path (including normalized path params), combining endpoint mappings from generated clients with request/response/example call data from wire tests.

Adds a Bun test suite with fixtures to validate the extractors across all three languages (plus edge cases like long Python tests, Java multi-line signatures, and brace/comment handling) and wires up a GitHub workflow (test-js-actions.yml) to run bun install + bun test for JS-based actions via a matrix. Documentation and repo hygiene are updated via README entries and a root .gitignore for node_modules/.

Reviewed by Cursor Bugbot for commit b21c6d9. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread scripts/extract-code-examples.ts Outdated
Comment thread extract-code-examples/index.ts
Comment thread scripts/extract-code-examples.ts Outdated
Comment thread .github/workflows/extract-code-examples.yml Outdated
Comment thread .github/workflows/extract-code-examples.yml Outdated
Comment thread scripts/extract-code-examples/index.ts Outdated
Comment thread scripts/extract-code-examples/index.ts Outdated
Comment thread extract-code-examples/index.ts
@gavinsharp gavinsharp changed the title feat: add reusable extract-code-examples workflow feat: add extract-code-examples composite action May 11, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 86e2eaa. Configure here.

Comment thread extract-code-examples/index.ts Outdated
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 11, 2026
PhenoML/sdk-shared-actions#1 (which adds the composite action) hasn't
merged yet, so @main has no action.yml. Pin to the branch for now;
flip back to @main or a tagged release once it lands.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp and others added 13 commits May 11, 2026 18:24
Adds a reusable workflow that SDK repos (TS, Python, Java) can call to
extract Fern-generated code examples into code-examples.json.

The extraction script is multi-language and auto-detects the SDK
language from .fern/metadata.json, so all three SDKs share the same
script and workflow with no per-repo customization.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a bun:test suite with unit tests for pure helpers and end-to-end
tests for each language parser, using real SDK files (Summary client,
authtoken auth raw_client + wire test, RawAuthClient + AuthtokenAuthWireTest)
as fixtures. Also fixes three parser bugs surfaced during PR review:

- Python wire-test scanner anchored its "next test function" break to a
  trimmed line, so indented nested `def test_*` helpers prematurely ended
  body scanning. Anchor to the raw line instead.
- Java endpoint parser's brace-depth tracking counted braces inside string
  literals, char literals, and comments — JSON fragments could corrupt
  method boundary detection. Replaced with a lexer-aware helper that
  carries block-comment and text-block state across lines.
- Java method-detection regex used `[^>]+`, which couldn't match nested
  generics like `PhenomlClientHttpResponse<Optional<Foo>>`. Switched to
  greedy `.+` so it backtracks to the outermost closing `>`.

Also extracts a shared `pathMatchesTemplate` helper used by both the
Python WireMock lookup and the manifest builder, and adds a GitHub
Actions workflow that runs `bun run test` on push/PR.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each script now owns its package.json, lockfile, tests, and fixtures so
future scripts can have independent toolchains and deps without sharing
node_modules. Workflow paths updated accordingly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The test workflow now runs as a single-entry matrix over `matrix.script`
so future scripts can be added with a one-line entry rather than
duplicating job steps.

The extract-code-examples workflow's `ref: ${{ github.head_ref }}` was
empty outside `pull_request` events (e.g. when called from `push`
workflows), leaving checkout in detached HEAD and breaking the later
`git push`. Falls back to `github.ref_name` so checkout always lands on
a named branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two related parser fixes plus matching tests:

- pyExtractTestExamples capped each test body scan at 60 lines, silently
  truncating longer generated wire tests so verify_request_count / SDK
  calls past that point were missed. Scan to end-of-file and rely on the
  existing top-level `def test_*` break for the boundary.
- buildManifest's Java chain fallback keyed on lowercased-concat of chain
  segments, which could collapse different chains (e.g. ["agent",
  "prompts"] vs ["agentp", "rompts"]) onto the same key and silently map
  examples to the wrong endpoint. Detect collisions at index time, warn,
  and drop the ambiguous entries so lookup misses loudly.

Tests cover both: a synthetic long-body Python fixture pushes the SDK
call past line 60, and direct buildManifest tests assert both the
normal chain match and the collision-drop behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
In `workflow_call` context, `github.workflow_sha` semantics have been
historically ambiguous (sometimes resolves to the caller's commit, not
the reusable-workflow file's commit). Parse `github.workflow_ref`
instead — its "owner/repo/path@ref" form is documented and unambiguous,
and the post-@ ref maps directly to the revision the caller pinned.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the workflow_call reusable workflow with a composite action at
extract-code-examples/. Callers now do a one-liner from inside their
existing job (after their own actions/checkout) and decide for themselves
whether to commit the resulting code-examples.json.

Cleaner than the reusable-workflow + checkout dance: no parsing of
github.workflow_ref to locate the script, no double-checkout, no implicit
auto-commit. The action uses github.action_path to find its own bundled
script + lockfile.

Layout: each top-level directory in the repo is now one composite action
(extract-code-examples/, plus action.yml). Test workflow updated to
iterate that root structure as a matrix.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
javaExtractEndpoints recorded methodBraceDepth as the depth at the
method-signature line. When the signature spans multiple lines and the
opening `{` is on a later line, that recorded depth was *before* the
body brace — so closing the method only returned braceDepth *to*
methodBraceDepth, never *below* it, and the exit check never fired.
A non-API helper between two API methods could then overwrite the
first method's collected pathSegments / httpMethod before the next
methodMatch saved it, attributing the helper's bogus path+verb to the
preceding method.

Fix: when the methodMatch line itself doesn't include the body `{`
(delta == 0), anticipate the body open by recording methodBraceDepth
as braceDepth + 1. Required pairing the exit check with `else if` so
it doesn't fire on the same line the methodMatch just set the depth.

Tested with a new java-multiline fixture: two API methods on `/things`
separated by a private helper whose `.method("PUT", ...)` and
`.addPathSegments("wrong/path")` would silently corrupt the first
method's state under the old behavior. With the fix, both endpoints
extract correctly and "PUT /wrong/path" never appears.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Top-level README now just enumerates available actions and links to
each one's directory. Per-action README documents inputs, usage, and
development for that specific action.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- README: re-add reference to reusable workflows; future entries will
  cover both composite actions and reusable workflows.
- test.yml: comment on the action matrix to clarify it's for JS-based
  actions only — shell-only composites, Docker actions, and reusable
  workflows have their own validation patterns.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
"test" was too generic for what's specifically a JS-actions test
matrix. Future workflows for other action types (Docker, reusable
workflows, shell composites) will have their own names.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Just enumerate the actions; per-action READMEs cover usage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
pyExtractEndpoints only called normalizePath on the extracted httpPath,
so a camelCase path parameter like /users/{userId} would survive as-is
in the manifest while TS/Java parsers always emit snake_case. Today's
Fern Python generator uses snake_case so this is benign, but the
parsers were inconsistent and could silently diverge if that changes.

Also chains normalizePathParams() so the Python path goes through the
same two-stage normalization the WireMock map lookup already uses.

Test: synthetic users/raw_client.py with f"users/{jsonable_encoder(userId)}"
asserts the manifest key is /users/{user_id}.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@gavinsharp
gavinsharp force-pushed the extract-code-examples branch from b7b201e to b21c6d9 Compare May 11, 2026 22:26
@gavinsharp
gavinsharp merged commit 6f75653 into main May 11, 2026
2 checks passed
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 11, 2026
PhenoML/sdk-shared-actions#1 has merged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 13, 2026
PhenoML/sdk-shared-actions#1 (which adds the composite action) hasn't
merged yet, so @main has no action.yml. Pin to the branch for now;
flip back to @main or a tagged release once it lands.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 13, 2026
PhenoML/sdk-shared-actions#1 has merged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 15, 2026
PhenoML/sdk-shared-actions#1 (which adds the composite action) hasn't
merged yet, so @main has no action.yml. Pin to the branch for now;
flip back to @main or a tagged release once it lands.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 15, 2026
PhenoML/sdk-shared-actions#1 has merged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gavinsharp added a commit to PhenoML/phenoml-ts-sdk that referenced this pull request May 15, 2026
* ci: add code examples extraction workflow

Uses the shared workflow from PhenoML/github-actions to extract
structured code examples from wire tests and client source, producing
a code-examples.json manifest keyed by HTTP method + path. Runs on
every PR and commits the manifest back to the branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: inline extraction workflow instead of using shared action

The shared workflow in PhenoML/github-actions can't be referenced from
this public repo since github-actions is private. Inlining the workflow
and script directly here until that's resolved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: sync extraction script and switch workflow to bun

Syncs to the latest version of the script from the pending shared action
(PhenoML/github-actions PR #2) and switches the workflow to use Bun with
an isolated scripts/ package, matching the shared action's setup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: use shared extract-code-examples workflow

Replaces the inlined workflow and bundled extraction script with a
thin caller that delegates to PhenoML/sdk-shared-actions. The script
and its dependencies now live once in the shared repo and are reused
by all SDK repos.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: switch to extract-code-examples composite action

The shared workflow was reworked into a composite action. The caller
now sets up its own checkout, calls the action, and handles the
commit-back itself.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: pin extract-code-examples action to branch until shared PR merges

PhenoML/sdk-shared-actions#1 (which adds the composite action) hasn't
merged yet, so @main has no action.yml. Pin to the branch for now;
flip back to @main or a tagged release once it lands.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: pin extract-code-examples action back to @main

PhenoML/sdk-shared-actions#1 has merged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: cancel in-progress extract runs on PR updates

Adds a concurrency block keyed on github.head_ref so that rapid PR
pushes cancel earlier runs of this workflow, preventing two runs from
racing to push code-examples.json back to the branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: skip extract-code-examples for fork PRs

On pull_request from a fork, github.head_ref names the contributor's
branch which doesn't exist in the base repo, and we couldn't push the
manifest back to a fork anyway. Skip the job entirely for fork PRs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: update code-examples.json

* chore: update code-examples.json

* chore: update code-examples.json

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant