Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,33 @@ jobs:
- name: Run Unit Tests
run: npm run test

# Verify the hand-written Zeebe gRPC facade in src/zeebe/ matches the
# upstream gateway.proto pinned in .zeebe-proto-version. Fails the build
# if any RPC, top-level message, or field defined in the proto is missing
# from the SDK. See MAINTAINER.md for how to refresh the proto and
# backfill new surfaces with `npm run emit:grpc-additions -- --apply`.
grpc-drift:
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6

- name: Use Node.js
uses: actions/setup-node@v6
with:
node-version: "22"

- name: Install dependencies
run: npm ci

- name: Verify zeebe.proto matches pinned upstream ref
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: npm run check:zeebe-proto

- name: Verify SDK facade covers all proto surfaces
run: npm run check:grpc-drift

# 8.8 self-managed integration tests. Uses only the public docker-compose
# stack — no secrets required. Runs on every push and every pull request
# (including fork PRs) so external contributors get PR feedback.
Expand Down
96 changes: 96 additions & 0 deletions .github/workflows/proto-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Proto sync (zeebe gateway.proto)

# Weekly job that fetches the latest upstream gateway.proto from
# camunda/camunda at the ref pinned in .zeebe-proto-version, regenerates
# any auto-emittable additive surface (Class 1-3 RPCs and new wire-type
# interfaces), and opens a pull request when there is drift.
#
# If the upstream ref has introduced a new Class 4 RPC (bespoke shaping
# required: nested *Request, file-bytes, real oneofs, or per-element
# variables), the emitter exits with code 3 and this workflow fails so a
# maintainer is alerted to design the facade by hand.

on:
schedule:
# Mondays at 04:00 UTC.
- cron: '0 4 * * 1'
workflow_dispatch: {}

jobs:
sync:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Check out the repo
uses: actions/checkout@v6

- name: Use Node.js
uses: actions/setup-node@v6
with:
node-version: "22"

- name: Install dependencies
run: npm ci

- name: Fetch upstream gateway.proto
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: npm run fetch:zeebe-proto

- name: Auto-emit Class 1-3 additive surface
# Hard-fails (exit 3) if any new RPC requires bespoke shaping.
run: npm run emit:grpc-additions -- --apply

- name: Verify generated TypeScript compiles
run: npx tsc --noEmit -p tsconfig.json

- name: Confirm drift detector is clean
run: npm run check:grpc-drift

- name: Read pinned ref
id: ref
run: |
ref=$(cat .zeebe-proto-version | tr -d '[:space:]')
sha=$(node -e "console.log(JSON.parse(require('fs').readFileSync('src/proto/.zeebe-proto-source.json','utf8')).sha.slice(0,12))")
echo "ref=$ref" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"

- name: Open pull request
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: chore/zeebe-proto-sync
base: main
commit-message: |
chore(zeebe): sync gateway.proto from camunda/camunda@${{ steps.ref.outputs.ref }}

Auto-generated by .github/workflows/proto-sync.yml.
Pinned ref: ${{ steps.ref.outputs.ref }} @ ${{ steps.ref.outputs.sha }}

Refs #760
title: "chore(zeebe): sync gateway.proto from camunda/camunda@${{ steps.ref.outputs.ref }}"
body: |
Automated weekly sync of `src/proto/zeebe.proto` from
`camunda/camunda@${{ steps.ref.outputs.ref }}`
(commit `${{ steps.ref.outputs.sha }}`).

New top-level wire-type interfaces, ZBGrpc method signatures,
and Class 1-3 facade methods were auto-emitted by
`scripts/emit-grpc-additions.mjs`.

**Maintainer checklist:**
- [ ] Review the auto-emitted region(s) (search for
`Auto-emitted by scripts/emit-grpc-additions.mjs`)
- [ ] Splice in any newly-reported optional fields on
existing wire-type interfaces (printed by the emitter
in `--print` mode; not auto-applied)
- [ ] If a new RPC needs bespoke shaping (Class 4), this
workflow will have failed instead — design the facade
by hand on a separate branch
- [ ] Update `CHANGELOG.md` if the additions warrant it
labels: |
zeebe
proto-sync
delete-branch: true
1 change: 1 addition & 0 deletions .zeebe-proto-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
stable/8.9
9 changes: 9 additions & 0 deletions AGENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,15 @@ See [Search endpoints](#search-endpoints) above. Summary:
3. Add a client factory method to the `Camunda8` class.
4. Export the client from `src/index.ts`.

### Updating the Zeebe gRPC facade

The `ZeebeGrpcClient` facade in `src/zeebe/` mirrors the upstream Zeebe `gateway.proto`. The local copy lives at `src/proto/zeebe.proto` and the upstream ref is pinned in `.zeebe-proto-version`.

- `npm run check:grpc-coverage` — see what (if anything) has drifted between the proto and the facade.
- `npm run emit:grpc-additions -- --apply` — auto-backfill new wire-types, ZBGrpc method signatures, and Class 1–3 facade methods. Hard-fails on Class 4 RPCs (nested `*Request`, `bytes`, real oneofs, per-element variables) — those need a hand-written facade.

A weekly workflow (`.github/workflows/proto-sync.yml`) runs the same pipeline and opens a PR. See **MAINTAINER.md → "Sync the Zeebe gateway.proto"** for the full procedure, including how to bump the pinned ref.

---

## Coding Guidelines
Expand Down
52 changes: 52 additions & 0 deletions MAINTAINER.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ All workflows live under [.github/workflows/](.github/workflows/).
| `backport.yml` | PR closed, `/backport` comment | Creates backport PRs (e.g. `main` → `stable/8.8`) via `korthout/backport-action`. |
| `integration-test-matrix.yml` | scheduled (02:00 UTC), manual dispatch | Cross-version client × server compatibility matrix. Runs **on a stable branch** and tests every released SDK version against every server version ≥ current minor. |
| `integration-test-matrix-trigger.yaml` | scheduled (02:00 UTC) | Fans out the matrix run to each active stable branch (uses `gh workflow run --ref stable/<x.y>`). |
| `proto-sync.yml` | scheduled (Mon 04:00 UTC), manual dispatch | Pulls latest Zeebe `gateway.proto` from the pinned upstream ref, runs the auto-emitter, and opens a PR if anything changed. See [§7 — Sync the Zeebe gateway.proto](#sync-the-zeebe-gatewayproto). |

### CI vs. Release: why feature branches and `main`/`stable/**` are separated

Expand Down Expand Up @@ -193,6 +194,54 @@ Re-run the `tag-and-publish` job from the Actions UI. semantic-release is idempo

Do **not** add an `NPM_TOKEN` secret. Verify on npmjs.com that `@camunda8/sdk` lists this repo + `release.yml` as a Trusted Publisher.

### Sync the Zeebe gateway.proto

The hand-written gRPC facade in `src/zeebe/` mirrors the upstream Zeebe `gateway.proto`. The local copy lives at `src/proto/zeebe.proto` and the upstream ref is pinned in `.zeebe-proto-version` (currently `stable/8.9`).

#### Pipeline at a glance

| Script | Purpose |
| -------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `scripts/fetch-zeebe-proto.mjs` | Resolve pinned ref → SHA, fetch `gateway.proto` from `camunda/camunda`, write metadata. |
| `scripts/check-grpc-coverage.mjs` | Diff the proto against the facade. Reports missing RPCs, messages, and fields. Classifies each RPC 1–4. |
| `scripts/emit-grpc-additions.mjs` | Auto-emit Class 1–3 additive surfaces. **Hard-fails on Class 4** (bespoke shaping required). |

npm script aliases:

```bash
npm run fetch:zeebe-proto # refresh src/proto/zeebe.proto
npm run check:zeebe-proto # CI: fail if local copy drifts from pinned upstream
npm run check:grpc-coverage # human-readable gap report
npm run check:grpc-drift # CI: exit non-zero on any gap
npm run emit:grpc-additions # print proposed additive code
npm run emit:grpc-additions -- --apply # write additive code into source files
```

#### Bumping the pinned proto ref

1. Edit `.zeebe-proto-version` (e.g. `stable/8.9` → `stable/9.0`).
2. `npm run fetch:zeebe-proto` — refreshes `src/proto/zeebe.proto`.
3. `npm run check:grpc-coverage` — see what changed.
4. `npm run emit:grpc-additions -- --apply` — auto-backfill Class 1–3 surfaces. If this exits with code 3, a new RPC needs a Class 4 (bespoke) facade designed by hand; do that on a separate commit before re-running.
5. Splice in any newly-reported optional fields on existing wire-types (the emitter prints these in `--print` mode but does not auto-apply — placement inside an existing interface body is not safe to automate).
6. `npx tsc --noEmit -p tsconfig.json` and `npm run check:grpc-drift` — both must be clean.
7. Commit with a `feat(zeebe):` message; flag any breaking changes.

#### Weekly automation

`.github/workflows/proto-sync.yml` runs the above pipeline every Monday at 04:00 UTC (and on manual dispatch) and opens a PR titled `chore(zeebe): sync gateway.proto from camunda/camunda@<ref>`. The PR body includes a maintainer checklist for the manual splice step. The workflow fails (and does not open a PR) if the emitter detects a Class 4 RPC.

#### Shape classes (request-shape classifier)

| Class | Trigger | Facade template |
| ----- | -------------------------------------------------------------------- | -------------------------------------------------- |
| 1 | Pure pass-through (no `string variables`, no `operationReference`) | Direct forward to `*Sync()`. |
| 2 | Has optional `operationReference` | Spread + `operationReference?.toString()`. |
| 3 | Has `string variables` field | Spread + `losslessStringify(variables)` + `tenantId ?? this.tenantId`. |
| 4 | Nested `*Request` field, `bytes`, real oneofs, or per-element variables | Hand-written by a maintainer. |

Synthetic oneofs from proto3 `optional` fields (named `_fieldname`) are filtered out and do not trigger Class 4.

---

## 8. Cross-references
Expand All @@ -203,3 +252,6 @@ Do **not** add an `NPM_TOKEN` secret. Verify on npmjs.com that `@camunda8/sdk` l
- [.github/workflows/ci.yml](.github/workflows/ci.yml) — PR / feature-branch CI
- [.github/workflows/integration-test-matrix.yml](.github/workflows/integration-test-matrix.yml) — nightly compatibility matrix
- [.github/workflows/backport.yml](.github/workflows/backport.yml) — automated backports
- [.github/workflows/proto-sync.yml](.github/workflows/proto-sync.yml) — weekly Zeebe `gateway.proto` sync
- [.zeebe-proto-version](.zeebe-proto-version) — pinned upstream ref for the Zeebe gateway proto
- [scripts/fetch-zeebe-proto.mjs](scripts/fetch-zeebe-proto.mjs), [scripts/check-grpc-coverage.mjs](scripts/check-grpc-coverage.mjs), [scripts/emit-grpc-additions.mjs](scripts/emit-grpc-additions.mjs) — proto-sync pipeline
5 changes: 5 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@
"compile": "tsc --project tsconfig.json",
"docs": "rm -rf ./docs && typedoc",
"docs:watch": "rm -rf ./docs && typedoc --watch",
"fetch:zeebe-proto": "node scripts/fetch-zeebe-proto.mjs",
"check:zeebe-proto": "node scripts/fetch-zeebe-proto.mjs --check",
"check:grpc-coverage": "node scripts/check-grpc-coverage.mjs",
"check:grpc-drift": "node scripts/check-grpc-coverage.mjs --fail-on-gap",
"emit:grpc-additions": "node scripts/emit-grpc-additions.mjs",
"generate:grpc": "grpc_tools_node_protoc --plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts --js_out=import_style=commonjs,binary:./src/generated --grpc_out=./src/generated --ts_out=./src/generated -I ./src/proto ./src/proto/*.proto",
"sm:start": ". env/sm-mt.env && docker compose --env-file docker/.env -f docker/docker-compose-multitenancy.yaml -f docker/docker-compose-modeler.yaml up -d && docker/suppress-es-deprecation.sh",
"sm:stop": "docker compose -f docker/docker-compose-multitenancy.yaml -f docker/docker-compose-modeler.yaml down && docker/cleanup-volumes.sh",
Expand Down
Loading
Loading