-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add typespec sync GH action workflow to automate emitted code merge #37806
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 32 commits
67ac255
3a8d023
7d8b0d1
c4dceb6
d9c4289
3d64992
f43b273
3a99ffb
e4c220e
06ecdaa
ebe0af3
e65b37e
605fd4a
c96387e
d09499f
2279075
91ac9f3
2fb3955
bdd3411
a6bff21
adeffe1
f3b2df4
a0202d5
78d7037
c1bb566
69379cd
1e6861a
cdbf6f6
279cd59
734c255
ea0bd01
aac05e9
db74061
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| # Workflow: TypeSpec Sync | ||
| # | ||
| # Monitors the TypeSpec spec repo for new commits and regenerates the | ||
| # library client if the spec has changed. Changes are committed to a | ||
| # configurable target branch. | ||
| # | ||
| # Required: "generate:client" script in target lib's package.json (and "generated" folder | ||
| # if "generate:client" script invokes "tsp-client update") | ||
| # | ||
| # Runs daily at midnight UTC and can be triggered manually. | ||
|
|
||
| name: TypeSpec Sync | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "0 0 * * *" | ||
|
|
||
| workflow_dispatch: | ||
| inputs: | ||
| package_path: | ||
| description: "Target folder (library) for synced changes" | ||
| required: false | ||
| default: "sdk/ai/ai-projects" | ||
| type: string | ||
| target_branch: | ||
| description: "Target branch for synced changes" | ||
| required: false | ||
| default: "feature/ai-projects-next" | ||
| type: string | ||
| source_branch: | ||
| description: "Source branch in the TypeSpec spec repo" | ||
| required: false | ||
| default: "feature/foundry-staging" | ||
| type: string | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
glharper marked this conversation as resolved.
|
||
|
|
||
| env: | ||
| TARGET_BRANCH: ${{ inputs.target_branch || 'feature/ai-projects-next' }} | ||
| SOURCE_BRANCH: ${{ inputs.source_branch || 'feature/foundry-staging' }} | ||
| PACKAGE_PATH: ${{ inputs.package_path || 'sdk/ai/ai-projects' }} | ||
|
|
||
| jobs: | ||
| sync-and-generate: | ||
| name: Check for spec updates and regenerate | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: true | ||
|
|
||
| node-version: "20.x" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is this correct?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. GH Copilot code review told me to make it node v20. |
||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: "24.x" | ||
|
|
||
| # ── Step 1: Ensure target branch exists ────────────────────── | ||
| - name: Ensure target branch exists | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| if git ls-remote --heads origin "refs/heads/$TARGET_BRANCH" | grep -q .; then | ||
| echo "Branch '$TARGET_BRANCH' already exists." | ||
| git checkout "$TARGET_BRANCH" | ||
| git pull origin "$TARGET_BRANCH" | ||
| else | ||
| echo "Branch '$TARGET_BRANCH' does not exist. Creating from main." | ||
| git checkout main | ||
| git pull origin main | ||
| git checkout -b "$TARGET_BRANCH" | ||
| git push origin "$TARGET_BRANCH" | ||
| fi | ||
|
|
||
| # ── Step 2: Parse tsp-location.yaml ────────────────────────── | ||
| - name: Parse tsp-location.yaml | ||
| id: parse | ||
| run: | | ||
| set -euo pipefail | ||
| cd "$PACKAGE_PATH" | ||
|
|
||
| SPEC_REPO=$(yq '.repo' tsp-location.yaml) | ||
| SPEC_DIR=$(yq '.directory' tsp-location.yaml) | ||
| CURRENT_COMMIT=$(yq '.commit' tsp-location.yaml) | ||
|
|
||
| echo "Spec repo: $SPEC_REPO" | ||
| echo "Spec directory: $SPEC_DIR" | ||
| echo "Current commit: $CURRENT_COMMIT" | ||
|
|
||
| echo "spec_repo=$SPEC_REPO" >> "$GITHUB_OUTPUT" | ||
| echo "spec_dir=$SPEC_DIR" >> "$GITHUB_OUTPUT" | ||
| echo "current_commit=$CURRENT_COMMIT" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # ── Step 3: Check latest commit in the spec repo ───────────── | ||
| - name: Check for new commits in spec repo | ||
| id: check | ||
| env: | ||
| SPEC_REPO: ${{ steps.parse.outputs.spec_repo }} | ||
| SPEC_DIR: ${{ steps.parse.outputs.spec_dir }} | ||
| CURRENT_COMMIT: ${{ steps.parse.outputs.current_commit }} | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| API_URL="https://api.github.com/repos/${SPEC_REPO}/commits" | ||
| API_URL+="?sha=${SOURCE_BRANCH}" | ||
| API_URL+="&path=${SPEC_DIR}" | ||
| API_URL+="&per_page=1" | ||
|
|
||
| echo "Querying: $API_URL" | ||
| RESPONSE=$(curl -sf "$API_URL") | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. GH aggressively rate limits anonymous calls to their APIs so this may fail.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's unfortunate for one curl every 24h. What would you suggest? |
||
| LATEST_COMMIT=$(echo "$RESPONSE" | jq -r '.[0].sha') | ||
|
|
||
| echo "Latest commit in spec repo: $LATEST_COMMIT" | ||
| echo "Current commit in tsp-location.yaml: $CURRENT_COMMIT" | ||
|
|
||
| if [ -z "$LATEST_COMMIT" ] || [ "$LATEST_COMMIT" = "null" ]; then | ||
| echo "::error::Failed to fetch latest commit from the spec repo." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ "$LATEST_COMMIT" = "$CURRENT_COMMIT" ]; then | ||
| echo "Commits match — no regeneration needed." | ||
| echo "needs_regeneration=false" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "New commit detected — regeneration required." | ||
| echo "needs_regeneration=true" >> "$GITHUB_OUTPUT" | ||
| echo "latest_commit=$LATEST_COMMIT" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| # ── Step 4a: Update tsp-location.yaml ──────────────────────── | ||
| - name: Update tsp-location.yaml with new commit | ||
| if: steps.check.outputs.needs_regeneration == 'true' | ||
| env: | ||
| LATEST_COMMIT: ${{ steps.check.outputs.latest_commit }} | ||
| run: | | ||
| set -euo pipefail | ||
| cd "$PACKAGE_PATH" | ||
|
|
||
| sed -i "s/^commit: .*/commit: $LATEST_COMMIT/" tsp-location.yaml | ||
|
|
||
| echo "Updated tsp-location.yaml:" | ||
| cat tsp-location.yaml | ||
|
|
||
| # ── Step 4b: Install dependencies and regenerate client ────── | ||
| - name: Install dependencies and regenerate client | ||
| if: steps.check.outputs.needs_regeneration == 'true' | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| npm install -g pnpm@latest-10 | ||
| npm install -g @azure-tools/typespec-client-generator-cli | ||
| pnpm install | ||
|
|
||
| cd "$PACKAGE_PATH" | ||
|
|
||
| npm run generate:client | ||
|
|
||
| # ── Step 5: Commit and push changes ────────────────────────── | ||
| - name: Commit and push changes | ||
| if: steps.check.outputs.needs_regeneration == 'true' | ||
| env: | ||
| LATEST_COMMIT: ${{ steps.check.outputs.latest_commit }} | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| git config user.email "AzureSDKPipelineBot@microsoft.com" | ||
|
glharper marked this conversation as resolved.
Outdated
|
||
| git config user.name "Azure SDK Pipeline Bot" | ||
|
|
||
| git add -A | ||
|
|
||
| if git diff --cached --quiet; then | ||
| echo "Regeneration produced no changes — nothing to commit." | ||
| exit 0 | ||
| fi | ||
|
|
||
| git commit -m "Regenerate $(basename "$PACKAGE_PATH") client from spec commit $LATEST_COMMIT" | ||
| git push origin "$TARGET_BRANCH" | ||
|
|
||
| echo "Changes committed and pushed to '$TARGET_BRANCH'." | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,11 @@ | ||
| # Release History | ||
|
|
||
| ## 2.0.2 (Unreleased) | ||
|
|
||
| ### Features Added | ||
|
|
||
| - TBD | ||
|
|
||
| ## 2.0.1 (2026-03-13) | ||
|
|
||
| ### Bugs Fixed | ||
|
|
||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@deyaaeldeen this looks to be scoped to just the ai projects do are there plans to do this for other sdks?
@glharper I'm also curious why we are choosing to use this approach over the usual pipeline we use to create PRs when specs change? cc @raych1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not that I am aware of. I am also curious about whether the auto PR pipeline could help here but I am not familiar with it. /cc @MaryGao
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@deyaaeldeen this is the JS pipeline. It's used in both the spec PR and release scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@weshaggard This adds the "generate:client" step, where the emitted code is merged with existing custom code. The issue with the "push from TypeSpec repo" approach is that each language has its own approach to dealing with custom code in TypeSpec-based libs, as well as post-emitter steps (dealing with idiosyncratic lang-specific emitter behavior).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
generate:clientstep runs 'tsp-client update' which is essentially same as the command - 'tsp-client init --update-if-exists' ran in the pipeline.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we would use tsp-client update for generation and we would also run a series of necessary commands e.g code build, format, sample publish, code snippet update, customization apply etc. Let us know if there are some commands need to tweak.