Skip to content

Experimental APIs [WiP]#10072

Draft
stephanos wants to merge 13 commits intomainfrom
stephanos/experimental-apis
Draft

Experimental APIs [WiP]#10072
stephanos wants to merge 13 commits intomainfrom
stephanos/experimental-apis

Conversation

@stephanos
Copy link
Copy Markdown
Contributor

@stephanos stephanos commented Apr 25, 2026 •
edited
Loading

What changed?

This PR prototypes an experimental API development flow without long-lived Temporal feature branches.

It adds:

  • frontend variant registration behind frontend.apiVariant
  • generated experimental api-go modules for additive RPCs (ping, tinker)
  • generated wrappers for new fields on existing stable messages (fairness)
  • functional coverage in Temporal for additive RPC variants and the fairness wrapper round-trip

Why?

Developing in-progress APIs across api, api-go, and temporal currently requires feature branches, repeated rebases, and a large final merge. This change explores a smaller incremental path where Temporal can depend on temporary experimental api-go modules until the stable API lands.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

Local verification included:

  • go test ./cmd/experimentgen in api-go
  • go test ./... in the generated ping, tinker, and fairness experiment modules
  • go test -tags 'test_dep experimental' ./service/frontend/services
  • go test -tags 'test_dep experimental' ./tests/testcore ./tests -run 'TestExperimentalApi_(Ping|Tinker|Stable|FairnessWrapper)'
  • go test -c -o /dev/null ./tests -tags 'disable_grpc_modules,test_dep,experimental'
  • make fmt-imports
  • make lint-code

Potential risks

This is prototype-only infrastructure and increases complexity in frontend registration, test wiring, and module generation. The wrapper path intentionally relies on protobuf unknown fields, so it is best suited to additive field experiments and should stay temporary until the stable API is merged.

@stephanos stephanos changed the title Stephanos/experimental apis Experimental APIs [WiP] Apr 25, 2026
@stephanos stephanos force-pushed the stephanos/experimental-apis branch 28 times, most recently from 24923a8 to 1e06e05 Compare April 25, 2026 23:06
@stephanos stephanos force-pushed the stephanos/experimental-apis branch from 1e06e05 to eb1bde8 Compare April 25, 2026 23:30
stephanos and others added 12 commits April 30, 2026 14:58
@stephanos @claude
Cut over server to generated api-go experimental files
334dc29 
Replaces the hand-authored github.com/temporalio/api-go/experimental/example
separate module with the generated files now in go.temporal.io/api under
//go:build experimental.

Changes:
- go.mod: remove experimental/example dep; add replace directive pointing
  to local api-go with the generated files
- api_example.go: workflowservice.EchoRequest/Response and
  workflowservice.ExampleWorkflowService_ServiceDesc replace expexample.*
- experimental_api_test.go: NewExampleWorkflowServiceClient, EchoRequest,
  WORKFLOW_ID_CONFLICT_POLICY_FOO now from go.temporal.io/api directly;
  MessageOverlay test skipped until experimental_field annotations are added

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Add expguard: detect experimental unknown fields at stable frontend b…
28e0f37 
…oundary

Creates common/expguard with a single Check() function: looks at
msg.ProtoReflect().GetUnknown() and softassert.Fails if non-empty.
One branch, same behavior in tests and prod — no modes, no reflection
walk, no generated sidecar.

Wires it as a frontend gRPC unary interceptor (ExperimentalGuardInterceptor)
that fires only when no API variant is active. When a variant IS active,
experimental clients are expected and the check is skipped.

The compile-time boundary (//go:build experimental) already prevents
stable server code from constructing experimental values. expguard is
defense-in-depth for the remaining case: an experimental client sends
unknown fields to a stable server, and those bytes ride through the
gRPC stack toward the history service.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Re-enable TestExperimentalApi_MessageOverlay with generated overlay t…
2b1f8bf 
…ypes

Tests Set/Get/Clear on StartWorkflowExecutionRequestOverlay, now
generated into go.temporal.io/api/workflowservice/v1 via experimentgen.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Remove stale comment from go.mod
3b8cb99 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Point go.temporal.io/api at remote branch pseudo-version
b892920 
Replaces the local path replace directive with a proper pseudo-version
resolved from the stephanos/experimental-apis branch of api-go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Remove local replace directive for go.temporal.io/api
bd77dcf 
go.sum now has the correct entries for the remote pseudo-version.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Add server-side demos for experimental enum value and overlay field
b3bb903 
Two new subtests in TestExperimentalApi_Example send each experimental
annotation type through a real cluster, proving all four reach the handler:

  experimental_method        — Echo RPC responds
  experimental_message       — EchoRequest/EchoResponse used by Echo
  experimental_enum_value    — WORKFLOW_ID_CONFLICT_POLICY_FOO sent via StartWorkflow
  experimental_field/overlay — foo_text overlay sent via StartWorkflow

The enum and overlay subtests assert NotEqual(Unimplemented) rather than
NoError: the server may reject the unknown conflict policy value (1000)
as invalid, but it must not reject it at the gRPC layer — proving the
value crossed the boundary and reached the handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Show production usage of all four experimental annotation types
0c8b1bc 
api_example.go now demonstrates how a real feature uses each type in
server code:

  experimental_method/message — exampleHandler.Echo (unchanged)
  experimental_enum_value     — exampleWorkflowWrapper.StartWorkflowExecution
                                reads WORKFLOW_ID_CONFLICT_POLICY_FOO and
                                returns InvalidArgument
  experimental_field/overlay  — same wrapper calls
                                GetStartWorkflowExecutionRequestOverlay to
                                read foo_text before delegating to stable handler

The wrapper embeds WorkflowServiceServer and only overrides the one method
it needs, relying on the stable handler for everything else.

Tests updated to assert codes.InvalidArgument for the enum subtest now
that the handler explicitly handles the FOO value.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Add hook pattern for mixing stable and experimental code
cd3fc53 
Demonstrates two patterns for safe stable/experimental mixing:

Pattern 1 — frontend wrapper (api_example.go, existing):
  Intercepts at the gRPC boundary via exampleWorkflowWrapper. Stable
  workflow_handler.go is never modified. Right for API-boundary changes.

Pattern 2 — hook interface (service/frontend/hook/hook.go, new):
  Stable code defines StartWorkflowHook with a no-op default and calls
  hook.StartWorkflow.PreStart(ctx, req) without a build tag. Experimental
  code registers exampleStartWorkflowHook in init(). Right for injecting
  behaviour deeper in the stack without touching stable files.

The hook package sits between frontend and frontend/services to avoid
the import cycle that would arise if hooks lived in either.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Simplify experimental mixing: use build-tag-gated function pair
0269e1c 
Replaces the hook interface/variable pattern with the same approach
already used in services/registry_default.go + registry_experimental.go:

  workflow_handler.go            — one added call: startWorkflowExperimentalCheck(req)
  workflow_handler_default.go    — //go:build !experimental  → returns nil
  workflow_handler_experimental.go — //go:build experimental → real logic

No interface, no global variable, no registration. Stable builds get an
inlined nil return. Experimental builds get the check. One call-site in
the stable file; two files for the two worlds.

api_example.go simplified: enum value and overlay handling moved to
workflow_handler_experimental.go where they belong.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Remove all build tags; use experimental module (two-module approach)
eb5fa43 
No //go:build experimental anywhere. Experimental types come from
github.com/temporalio/api-go/experimental — a plain Go dependency.
Stable go.temporal.io/api stays clean.

Changes:
- registry_default.go + registry_experimental.go deleted; registry
  inlined into register.go (no build tag needed)
- api_example.go: removed build tag, imports exp{enums,workflowservice}
  from the experimental module directly
- workflow_handler.go: reverted (no experimental check call)
- workflow_handler_default.go + workflow_handler_experimental.go deleted
- Makefile: removed experimental from ALL_TEST_TAGS (no longer needed)
- Tests: import expworkflowservice/expenums from experimental module

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stephanos @claude
Fix make build-tests: add experimental dep, fix enum type reference
4d65b5f 
- go.mod: re-add github.com/temporalio/api-go/experimental dep (removed
  by go mod tidy since it wasn't in go.sum yet)
- experimental_api_test.go: use enumspb.WorkflowIdConflictPolicy for the
  cast — the experimental module exports only the constant, not the type
  (which lives in the stable enums package)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@stephanos