[Proposal][Design] Per-Operation Upstream Override on OperationRequest

[mehara-rothila]

Background

The current OperationRequest schema does not support per-operation upstream routing. All operations of a REST API share the API-level upstream.main (and optionally upstream.sandbox), as defined in openapi.yaml.

In real-world deployments, a single API often fronts a microservice cluster where each resource maps to a different backend service. Today this requires creating a separate REST API per backend, which fragments the API contract and breaks the "one logical API, many implementations" model.

This discussion proposes adding an optional upstream field on OperationRequest so a single REST API can route its operations to different backends.

It builds on prior decisions in:

• Discussion API YAML Definition with Upstream Definition and Main/Sandbox Upstreams #369 — upstreamDefinitions schema: defined the named upstream definition system with basePath and reusable upstream clusters under upstreamDefinitions.
• Discussion [Proposal] Rewrite Path and Dynamic Endpoint - Resolve Conflicting Policies #1298 — Rewrite Path / Dynamic Endpoint conflict: resolved the clear_route_cache incompatibility by splitting responsibilities across an ext_proc filter (sets x-target-upstream and clears route cache) and a Lua filter (modifies :path/:method without clearing route cache). The route uses cluster_header pointing to x-target-upstream with request_headers_to_remove to strip the internal header before forwarding upstream.

---

Proposed schema

A new optional upstream field on OperationRequest. The shape is an OperationUpstream wrapper that mirrors the API-level Upstream shape with separate main and sandbox sub-fields. Each sub-field is an OperationUpstreamTarget — a strict subset of UpstreamDefinition containing only url and ref (mutually exclusive via oneOf). The auth sub-field is omitted entirely so per-op upstream credentials cannot be expressed in the schema (see Validation rules).

upstreamDefinitions:
  - name: user-service-cluster
    basePath: /api/v2
    upstreams:
      - url: http://user-service-a:5000
      - url: http://user-service-b:5000

upstream:
  main:
    url: http://default-backend:5000
  sandbox:
    url: http://default-sandbox:5000

operations:
  - name: listUsers
    request:
      method: GET
      path: /users
      upstream:                        # <-- NEW
        main:
          ref: user-service-cluster       # routes to the named cluster from #369
        sandbox:
          url: http://user-service-test:5000  # test backend for sandbox

  - name: getOrder
    request:
      method: GET
      path: /orders/{id}
      upstream:                        # <-- NEW
        main:
          url: http://order-service:5000   # direct URL override
        # sandbox omitted → falls back to API-level upstream.sandbox

  - name: healthCheck
    request:
      method: GET
      path: /health                    # no upstream → falls back to API-level

---

Schema details

OperationUpstream (wrapper)

OperationUpstream:
  type: object
  description: Per-operation upstream override. Mirrors the API-level Upstream shape.
  properties:
    main:
      $ref: "#/components/schemas/OperationUpstreamTarget"
      description: Override for the production vhost.
    sandbox:
      $ref: "#/components/schemas/OperationUpstreamTarget"
      description: Override for the sandbox vhost.

OperationUpstreamTarget (leaf)

OperationUpstreamTarget:
  type: object
  oneOf:
    - required: [ "url" ]
    - required: [ "ref" ]
  description: A single per-operation upstream target.
  properties:
    url:
      type: string
      format: uri
      description: Direct backend URL to route traffic to.
    ref:
      type: string
      description: Reference to a predefined upstreamDefinition.

Design notes:

• Both main and sandbox are optional. Either sub-field can be omitted independently.
• The auth sub-field is intentionally omitted from OperationUpstreamTarget. Operation-level credentials belong in header-injection or OAuth policies attached to the operation. Because auth is not in the schema at all, consumers cannot set it and the handler does not need a runtime carve-out.
• The ref field is reserved for forward compatibility with a future upstreamDefinitions registry at the Platform API layer. No such registry exists today, so any non-empty ref is rejected with 400 by the handler before DB write (see Validation rules). When the registry lands, this rejection lifts and refs resolve against the API's upstreamDefinitions list.

---

Resolution order

Per-operation upstream is environment-scoped. The gateway resolves independently for each vhost:

Environment	Priority 1 (per-op)	Priority 2 (fallback)
Main vhost	operation.request.upstream.main	API-level upstream.main
Sandbox vhost	operation.request.upstream.sandbox	API-level upstream.sandbox

Either sub-field can be omitted. An omitted sub-field falls back to the API-level upstream for that environment. This gives four valid configurations per operation:

upstream.main	upstream.sandbox	Main vhost behavior	Sandbox vhost behavior
Set	Set	Use per-op main	Use per-op sandbox
Set	Omitted	Use per-op main	Fall back to API sandbox
Omitted	Set	Fall back to API main	Use per-op sandbox
Omitted	Omitted	Fall back to API main	Fall back to API sandbox

---

Validation rules

• url and ref are mutually exclusive within a single target, enforced at the schema layer via oneOf.
• The ref field is rejected unconditionally by the Platform API handler: any non-empty ref under main or sandbox returns 400 Bad Request with sentinel ErrUpstreamRefNotSupported. The Platform API does not expose an upstreamDefinitions registry (the REST API schema has no upstreamDefinitions field), so refs have no target to resolve against. Schema keeps ref for forward compatibility; it will be accepted only when definitions are exposed at this layer.
• The per-op schema intentionally omits auth (see Schema details above).

---

End-to-end feature scope

platform-api (control plane)

• New OperationUpstream wrapper schema and OperationUpstreamTarget leaf schema added to openapi.yaml.
• OperationRequest schema gains the optional upstream field as a $ref to OperationUpstream.
• Generated wire type (api.OperationRequest.Upstream is *api.OperationUpstream) carries the field.
• Internal model (model.OperationRequest) reuses the existing UpstreamConfig type (which already has Main / Sandbox shape) for the per-op field. Auth is representable internally via UpstreamEndpoint.Auth but is never populated from wire input and is never emitted back to wire.
• Two new converter helpers in internal/utils/api.go round-trip the wrapper with nil-guarding:
  • operationUpstreamToModel(*api.OperationUpstream) *model.UpstreamConfig
  • upstreamConfigToOperationUpstream(*model.UpstreamConfig) *api.OperationUpstream
• Deployment YAML serializer (BuildAPIDeploymentYAML) emits per-op upstream when present and omits it otherwise.
• YAML import paths preserve per-op upstream on create and update so the field survives api.yaml round-trips.
• OpenAPI import merge (MergeRESTAPIDetails) stamps per-op upstream from the user body onto extracted operations by (Method, Path) key.
• Handler-level validation rejects any non-empty ref value (under main or sandbox) before DB write. A sentinel error ErrUpstreamRefNotSupported is returned as 400 Bad Request — the Platform API does not yet expose an upstreamDefinitions registry, so refs have no target to resolve against at this layer.
• Persistence: the per-op field lives inside the existing rest_apis.configuration JSON blob. No database migration required.

gateway-controller

• The controller's internal Operation config type gains the Upstream field as an OperationUpstream wrapper.
• Primary transformer path (pkg/transform/restapi.go Transform()):
  • The per-op loop derives separate cluster keys for main and sandbox vhosts.
  • op.Upstream.Main drives the main vhost route's cluster key.
  • op.Upstream.Sandbox drives the sandbox vhost route's cluster key.
  • Per-op routes pin traffic to a single backend, so UseClusterHeader and DefaultCluster are disabled for that route.
  • The sandbox override loop patches sandbox routes to the API-level sandbox cluster only for operations that do not have their own per-op sandbox override (op.Upstream.Sandbox != nil).
• Legacy translator path (pkg/xds/translator.go translateAPIConfig()):
  • The fallback route-building loop currently has no per-op upstream support — it builds main routes with mainClusterName and sandbox routes with sbClusterName for every operation. This proposal extends it with the same per-op cluster derivation logic as the primary path: read op.Upstream.Main / op.Upstream.Sandbox, register per-op clusters via the shared helper, and pin each vhost route to the resolved cluster. The sandbox loop gains the same guard: skip override when op.Upstream.Sandbox is set.
  • Implementation status: pending at the time of writing. Primary transformer path is the shipped surface; legacy path is the secondary fallback that needs to land before APIs that don't go through the primary transformer can use the feature.
• Cluster naming (synthetic clusters for url overrides).
  • For per-op url overrides, a synthetic Envoy cluster is registered.
  • Cluster keys are derived from a stable hash of the full resolved URL (op_<first-6-bytes-of-SHA256-in-hex>), so two operations pointing at the same backend collapse to a single cluster (xDS-snapshot friendly) and distinct URLs get distinct clusters.
  • The helper addPerOpUpstreamCluster is idempotent — identical URLs reuse the same cluster entry.
• ref resolution. Per-op ref values are resolved to a URL by resolveUpstreamURL, which looks up the referenced upstreamDefinition by name and returns its first upstream URL. That URL is then hashed with SHA-256, and the cluster key becomes op_<first-12-hex-chars> (e.g., op_a1b2c3d4e5f6). Two operations referencing the same definition collapse to the same cluster (dedup); distinct URLs never collide. Per-op upstreams never reuse the API-level scoped naming convention (upstream_<kind>_<apiId>_<name>), and EnvoyClusterName is intentionally left empty because per-op routes do not use the cluster_header dynamic-selection path.
• Cluster property inheritance. Synthetic clusters generated from per-op upstreams (whether via direct url or resolved ref) always have the same simple structure: BasePath from the URL path, a single endpoint from the URL host and port, and TLS enabled if the scheme is https. No other settings (auth, timeouts, load-balancing) are inherited from a referenced upstreamDefinition — only the resolved URL is used.
• Static cluster pinning vs. dynamic routing. Per-op upstream uses static cluster assignment, not the cluster_header dynamic-selection flow from [Proposal] Rewrite Path and Dynamic Endpoint - Resolve Conflicting Policies #1298. When a per-op override is present, restapi.go sets UseClusterHeader = false and DefaultCluster = "" for that route; the xDS translator emits RouteAction_Cluster with the per-op ClusterKey directly. The [Proposal] Rewrite Path and Dynamic Endpoint - Resolve Conflicting Policies #1298 ext_proc + Lua architecture (x-target-upstream header + cluster_header routing) is the mechanism for upstreamDefinitions-based policy-driven dynamic endpoints. The two modes coexist: routes without a per-op override may still use cluster_header when upstreamDefinitions are present; routes with a per-op override bypass that mechanism entirely and pin to their synthetic cluster.

gateway-runtime

• No runtime changes. The Lua filter's target_upstream_base_path lookup and the policy-engine's UseClusterHeader / DefaultCluster wiring from API YAML Definition with Upstream Definition and Main/Sandbox Upstreams #369 + [Proposal] Rewrite Path and Dynamic Endpoint - Resolve Conflicting Policies #1298 already route traffic correctly once the controller emits the right per-route cluster name and publishes it into upstreamDefinitionPaths.

---

Backward compatibility

• The upstream field is optional. Existing APIs that don't set it behave exactly as before.
• Storage is a JSON blob, so no schema migration touches existing rows.
• YAML export / import / update is a no-op for APIs that never adopt the field.
• The gateway controller falls back to mainUpstream.ClusterKey / sbUpstream.ClusterKey when no per-op override is set for that environment, matching current behavior.
• UpstreamDefinition is untouched — API-level upstream.main / upstream.sandbox still support auth exactly as before. Only the new per-op OperationUpstreamTarget schema omits it.
• The [Proposal] Rewrite Path and Dynamic Endpoint - Resolve Conflicting Policies #1298 ext_proc + Lua filter chain is untouched. Per-op upstream does not interact with the cluster_header / x-target-upstream dynamic routing path; it sets a static ClusterKey on the route, which the xDS translator emits as a direct RouteAction_Cluster.

---

Relevant code from #369 + #1298

For context, here are the layers that the per-cluster basePath rewriting and dynamic cluster selection from #369 + #1298 touch:

Layer	Component
gateway-runtime	policy-engine/internal/kernel/translator.go — the if execCtx.upstreamDefinitionPaths != nil block that looks up upstreamDefinitionPaths[*targetUpstreamName] and sets target_upstream_base_path on out.DynamicMetadata for the Lua filter
gateway-runtime	policy-engine/internal/xdsclient/handler.go — the if pathsRaw, ok := data["upstream_definition_paths"] block that hydrates rc.Metadata.UpstreamDefinitionPaths from xDS metadata
gateway-controller	pkg/xds/translator.go — upstreamDefPaths := make(map[string]string) block that builds the lookup from apiData.UpstreamDefinitions. Serialized into xDS metadata under upstream_definition_paths in pkg/policyxds/snapshot.go
gateway-controller	pkg/xds/translator.go — API-level upstream.ref resolution in resolveUpstreamCluster("main", ...)
gateway-controller	pkg/transform/restapi.go — the for _, op := range apiData.Operations loop in Transform(). Per-op cluster keys are derived independently for main and sandbox vhosts.
gateway-controller	pkg/xds/translator.go — fallback for _, op := range apiData.Operations loop in translateAPIConfig(). Per-op cluster derivation logic to be added here as part of this proposal (currently no per-op support; pending).
platform-api	OperationRequest schema in openapi.yaml — the OperationRequest: schema (where the new optional upstream field is added) and the new OperationUpstream: / OperationUpstreamTarget: schemas