feat(zeebe): autogenerate Zeebe gRPC facade from upstream proto#764
Open
jwulf wants to merge 6 commits into
Open
feat(zeebe): autogenerate Zeebe gRPC facade from upstream proto#764jwulf wants to merge 6 commits into
jwulf wants to merge 6 commits into
Conversation
Adds scripts/fetch-zeebe-proto.mjs which downloads the Zeebe gateway.proto from camunda/camunda at the ref pinned in .zeebe-proto-version (currently stable/8.9), records the resolved commit SHA in src/proto/.zeebe-proto-source.json, and refreshes src/proto/zeebe.proto. Also adds a --check mode for CI to fail if the local copy drifts from upstream. This is phase 0 of the proto-sync pipeline outlined in #760. Refs #760
Adds scripts/check-grpc-coverage.mjs which loads src/proto/zeebe.proto
via protobufjs and reports gaps against the hand-written facade in
src/zeebe/:
- RPCs missing from the ZBGrpc interface
- RPCs missing from the public ZeebeGrpcClient facade
- Top-level messages missing from interfaces-grpc-1.0.ts
- Fields on existing messages missing from their TS mirrors
Each missing RPC is classified into one of four shape-classes:
- Class 1 — pure pass-through
- Class 2 — operationReference normalisation only
- Class 3 — variables stringification
- Class 4 — bespoke shaping required (nested *Request, file-bytes,
real oneofs, or per-element variables)
Class 4 RPCs always require a human-designed facade. Classes 1-3 are
mechanically derivable from the proto and can be auto-emitted in a
later phase.
Synthetic oneofs from proto3 'optional' fields are filtered out and
do not trigger Class 4. Empty response messages, deprecated messages,
and inline-parameter request shapes are excluded from the missing-
messages report.
Refs #760
One-time backfill to bring the hand-written Zeebe gRPC facade in line
with the upstream gateway.proto pinned at camunda/camunda@stable/8.9.
Changes:
- interfaces-grpc-1.0.ts:
- New optional fields on existing wire types (ActivatedJob,
ActivateJobsRequest, StreamActivatedJobsRequest, CompleteJobRequest,
CreateProcessInstanceRequest/Response, CreateProcessInstanceWith-
ResultResponse, EvaluateDecisionResponse, EvaluatedDecision,
TopologyResponse, BrokerInfo, SetVariablesRequest,
ModifyProcessInstanceRequest, BroadcastSignalResponse).
- New top-level types: UserTaskProperties, JobResult,
JobResultCorrections, JobResultActivateElement, StringList,
ProcessInstanceCreationRuntimeInstruction,
TerminateProcessInstanceInstruction, MoveInstruction,
SetVariablesResponse, BatchOperationCreatedResult,
DeleteResourceResponse, EvaluateConditionalRequest,
ProcessInstanceReference, EvaluateConditionalResponse.
- New enums: TenantFilter, JobKind, ListenerEventType,
BatchOperationTypeEnum.
- interfaces-1.0.ts: extend ZBGrpc with evaluateConditionalSync; switch
deleteResourceSync return type from Record<string, never> to
DeleteResourceResponse.
- ZeebeGrpcClient.ts: add public evaluateConditional() (Class 3 facade)
and update deleteResource() return type to DeleteResourceResponse.
The drift detector (scripts/check-grpc-coverage.mjs) reports
"OK — no gaps detected." after this commit.
BREAKING CHANGE: ZeebeGrpcClient.deleteResource() now returns
Promise<DeleteResourceResponse> instead of Promise<Record<string, never>>.
Existing callers that ignored the return value are unaffected; callers
that destructured the previously-empty object should now receive
{ resourceKey } and, when deleteHistory was requested for a process
definition, a batchOperation handle.
Refs #760
Adds scripts/emit-grpc-additions.mjs which mechanically derives the
additive parts of the Zeebe gRPC facade from src/proto/zeebe.proto.
For each gap reported by check-grpc-coverage:
- new top-level wire-type interfaces are emitted into
interfaces-grpc-1.0.ts (proto field comments preserved as JSDoc;
int64-family typed as string to match the existing convention)
- new ZBGrpc method signatures are inserted into the ZBGrpc
interface in interfaces-1.0.ts
- new public facade methods are inserted into ZeebeGrpcClient.ts
using one of three templates:
Class 1 — pure pass-through
Class 2 — operationReference?.toString() normalisation
Class 3 — losslessStringify(variables) + tenantId default
- new fields on existing wire-type interfaces are printed for the
maintainer to splice in by hand (precise placement inside an
existing interface body is not safe to automate)
The script HARD FAILS (exit 3) if any new RPC requires bespoke (Class
4) shaping — nested *Request fields, file-bytes, real oneofs, or
per-element variables. Those facades must be hand-written.
Modes:
--print (default) emit proposed source to stdout
--apply write new interfaces, ZBGrpc signatures, and facade
methods into the source files (additions are appended in
a clearly-marked auto-emitted region)
--json emit the work list as JSON for downstream tooling
Wired into npm scripts:
npm run check:grpc-coverage # human report
npm run check:grpc-drift # CI: exit non-zero on any gap
npm run emit:grpc-additions # print proposed additive code
Verified end-to-end against the current 8.9 proto by removing
StringList, evaluateConditionalSync, and the evaluateConditional
facade method, then re-applying via --apply: the output compiled
cleanly under tsc and the drift detector reported zero gaps.
Refs #760
Two CI integrations for the Zeebe gRPC proto-sync pipeline:
1. ci.yml — new `grpc-drift` job runs on every push (non-main) and
pull request. It verifies:
a. src/proto/zeebe.proto matches the upstream ref pinned in
.zeebe-proto-version (`npm run check:zeebe-proto`)
b. the SDK's hand-written facade in src/zeebe/ covers every RPC,
message, and field defined in the proto
(`npm run check:grpc-drift`, exits non-zero on any gap)
This blocks PRs that modify the proto without updating the facade,
and PRs that update the facade against a stale proto.
2. .github/workflows/proto-sync.yml — new weekly workflow (Mondays
04:00 UTC, plus workflow_dispatch). Fetches the latest upstream
gateway.proto at the pinned ref, runs the auto-emitter to backfill
Class 1-3 additive surfaces, verifies tsc + drift detector are
clean, and opens a pull request via peter-evans/create-pull-request.
The PR body includes a maintainer checklist for splicing in
newly-reported optional fields on existing wire-types (which the
emitter prints but does not auto-apply).
If a new RPC requires bespoke (Class 4) shaping, the emitter exits
with code 3 and the workflow fails — alerting a maintainer to
design the facade by hand.
Refs #760
- MAINTAINER.md §7: new "Sync the Zeebe gateway.proto" subsection covering pipeline scripts, npm aliases, the bump-the-pinned-ref procedure, the weekly proto-sync.yml workflow, and the four request shape-classes used by the auto-emitter. - MAINTAINER.md §2 (CI Workflows table): add a row for proto-sync.yml. - MAINTAINER.md §8 (Cross-references): link the workflow, the pinned-ref file, and the three pipeline scripts. - AGENT.md: short "Updating the Zeebe gRPC facade" subsection under Common Development Tasks pointing at the npm scripts and the MAINTAINER section. Refs #760
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements an end-to-end pipeline that keeps the hand-written Zeebe gRPC facade in sync with the upstream
gateway.protofromcamunda/camunda.Closes #760
What changed
Phase 0 — Proto fetch
scripts/fetch-zeebe-proto.mjsdownloadsgateway.protofromcamunda/camundaat the ref pinned in.zeebe-proto-version(currentlystable/8.9), records the resolved commit SHA insrc/proto/.zeebe-proto-source.json, and refreshessrc/proto/zeebe.proto. A--checkmode is provided for CI.Phase 1 — Drift detector
scripts/check-grpc-coverage.mjsparses the proto via protobufjs and reports gaps against the hand-written facade:ZBGrpcinterface or the publicZeebeGrpcClientfacadeinterfaces-grpc-1.0.tsEach missing RPC is classified into one of four shape-classes (1–3 are mechanically derivable; Class 4 requires a hand-designed facade).
Phase 2 — Additive-surface emitter
scripts/emit-grpc-additions.mjsmechanically derives additive surfaces (new types,ZBGrpcmethod signatures, and facade methods) for Class 1–3 RPCs. Class 4 RPCs cause a hard failure (exit 3) so a maintainer is alerted. Supports--print,--apply, and--jsonmodes.Phase 3 — 8.9 backfill
One-time backfill to bring the facade in line with
stable/8.9. Adds new wire types, enums, optional fields on existing types, and a newevaluateConditional()facade method.BREAKING CHANGE:
ZeebeGrpcClient.deleteResource()now returnsPromise<DeleteResourceResponse>instead ofPromise<Record<string, never>>. Callers that ignored the return value are unaffected.Phase 4 — CI integration
ci.yml— newgrpc-driftjob on push/PR verifies proto freshness and zero driftproto-sync.yml— weekly workflow (Monday 04:00 UTC) fetches latest proto, runs the emitter, and opens a PR automaticallyPhase 5 — Documentation
MAINTAINER.mdandAGENT.mdupdated with the proto-sync pipeline procedures and cross-references.npm scripts
npm run fetch:zeebe-protonpm run check:zeebe-protonpm run check:grpc-coveragenpm run check:grpc-driftnpm run emit:grpc-additions