[Proposal] Immutable Gateway v4.0.0 - File-Based, GitOps-Native Gateway Configuration

[renuka-fernando]

Summary

• Introduce immutable gateway mode for the API Platform Gateway, where API configurations are baked into the container image at build time and the management API is locked to read-only at runtime
• Add a new [immutable_gateway] config section in config.toml with enabled and artifacts_dir settings
• When enabled, the gateway controller rejects all mutating HTTP requests (POST, PUT, DELETE) to the management API across all resource types
• The gateway starts with an empty SQLite database and loads all API artifacts from the filesystem on startup, applying them through the existing service layer
• Introduce $env{key} syntax for resolving secrets from environment variables, complementing the existing $secret{key} syntax
• Support deterministic artifact UUIDs via the gateway.api-platform.wso2.com/artifact-id label in resource metadata, enabling API Key functionality with the Control Plane and label-based selection in Kubernetes

Motivation

• GitOps alignment: Production environments increasingly require that all configuration changes flow through version control, code review, and CI/CD pipelines rather than ad-hoc API calls to a running gateway
• Reproducibility: With mutable gateways, configuration drift can occur when multiple operators make changes. An immutable gateway ensures that every deployment is identical and reproducible from source
• Simplified operations: Operators do not need to manage secrets for the management API in production or worry about unauthorized runtime changes; the gateway is fully defined at build time
• Foundation for Immutable Gateway v4.0.0: The existing Immutable Gateway product (pre-v4) is being rebuilt on top of the API Platform Gateway, requiring native support for this mode of operation

Proposal

• Add a new configuration section [immutable_gateway] to config.toml that enables immutable mode and specifies the artifacts directory
• On startup, the gateway controller walks the artifacts directory recursively, discovers all YAML files, classifies them by kind, and applies them via the service layer in dependency order (independent resources first, dependent resources second); if any artifact fails validation, the gateway will not start
• The management API remains available for GET/list operations but returns 405 Method Not Allowed for all mutating operations when immutable mode is active
• Secrets are resolved from environment variables using $env{key} syntax instead of the existing $secret{key} mechanism, since there is no runtime API to create secrets

API Changes

No new API endpoints are introduced. The existing management API endpoints have modified behavior when immutable mode is enabled:

All of the following endpoints return 405 Method Not Allowed for POST, PUT, and DELETE when immutable mode is enabled:

• POST/PUT/DELETE /rest-apis and /rest-apis/{id}
• POST/PUT/DELETE /llm-providers and /llm-providers/{id}
• POST/PUT/DELETE /llm-proxies and /llm-proxies/{id}
• POST/PUT/DELETE /mcp-proxies and /mcp-proxies/{id}
• POST/PUT/DELETE /subscription-plans
• POST/PUT/DELETE /subscriptions
• POST/PUT/DELETE /secrets

GET endpoints remain fully functional for observability and debugging.

Configuration Changes

[immutable_gateway]
enabled = true
artifacts_dir = "/etc/api-platform-gateway/immutable_gateway/artifacts"

The corresponding environment variable overrides would follow the existing convention:

APIP_GW_IMMUTABLE_GATEWAY_ENABLED=true
APIP_GW_IMMUTABLE_GATEWAY_ARTIFACTS_DIR=/etc/api-platform-gateway/immutable_gateway/artifacts

Examples

Artifact YAML format (uses the Kubernetes-style resource format already supported by the gateway):

apiVersion: gateway.api-platform.wso2.com/v1alpha1
kind: RestApi
metadata:
  name: reading-list-api-v1
  labels:
    gateway.api-platform.wso2.com/artifact-id: "123e4567-e89b-12d3-a456" # NEW: optional; auto-generated if absent. Set explicitly for stable API Key association with Control Plane
spec:
  displayName: Reading List API
  version: v1.0
  context: /reading-list/$version
  upstream:
    main:
      url: "http://reading-list-api:8080"
  policies:
    - name: set-headers
      version: v1
      params:
        request:
          headers:
          - name: X-API-Key
            value: "$env{READING_LIST_API_KEY}" # NEW: resolved from container env vars
  operations:
    - method: GET
      path: "/books"

Dockerfile for building an immutable gateway image:

FROM ghcr.io/wso2/api-platform/gateway-controller:1.0.0

COPY ./artifacts /etc/api-platform-gateway/immutable_gateway/artifacts

Directory structure (flexible -- the controller walks all subdirectories):

artifacts/
├── rest-api/
│   ├── petstore-v1.yaml
│   └── orders-v2.yaml
├── llm-providers/
│   └── openai.yaml
├── llm-proxies/
│   └── chat-proxy.yaml
└── mcp-proxies/
    └── tools-proxy.yaml

Drawbacks

• Secret management limitations: The initial implementation only supports $env{key} for secrets, meaning all sensitive values must be injected as environment variables. This may not meet the security requirements of all organizations (environment variables can appear in process listings and crash dumps)
• Subscription and secret resources not supported via artifacts: Subscription plans, subscriptions, and secrets cannot be deployed through the artifacts directory in this initial version, limiting certain use cases
• Custom Dockerfile requirement: Users must write their own Dockerfile to build the immutable image. This adds friction compared to a single CLI command (though ap gateway image build support is planned for a future iteration)

[pubudu538]

Looks good. In the case of environment variables, users will have to manage a flat hierarchy here. In the immutable gateway v3.2.0, we had this format - https://mg.docs.wso2.com/en/latest/how-tos/endpoints/overriding-endpoint-information/overriding-endpoints-for-imported-apis/#overriding-endpoints-at-runtime. Basically API_ID has used as a prefix. Should we look for something like this? I think we can use the API handle here.

[renuka-fernando]

I've created a separate discussion on handling env var: #1688

Here, instead of predefined env vars, we can go with this approach, where the user has to define the backend ENV var in the RestAPI YAML and then set the ENV var on the Gateway Controller container. So then user can define their own ENV variable.

Ex:

apiVersion: gateway.api-platform.wso2.com/v1alpha1
kind: RestApi
metadata:
  name: reading-list-api-v1
spec:
  context: /reading-list/v1
  upstream:
    main:
      url: '{{ env "BACKEND_URL" | default "http://localhost:8080" }}'
    auth:
      # secret is always redacted in dumps automatically
      apiKey: '{{ secret "backend-api-key" }}'
      # env var holding a sensitive value — explicitly marked for redaction
      token: '{{ env "BEARER_TOKEN" | redact }}'
  policies:
    - name: set-headers
      version: v1
      params:
        request:
          headers:
            - name: Authorization
              value: 'Bearer {{ secret "oauth-token" }}'
            - name: X-Log-Level
              value: '{{ if env "DEBUG" }}debug{{ else }}info{{ end }}'

wdyt?

[pubudu538]

+1.

[renuka-fernando]

A couple of implementation constraints worth calling out:

1. SQLite only in immutable mode: Since the immutable gateway starts with a fresh DB on every boot, only SQLite should be supported. If a non-SQLite database is configured alongside immutable_gateway.enabled = true, the gateway controller should print a clear error and exit rather than silently proceeding or behaving unexpectedly.

2. Delete SQLite DB files on startup: When immutable mode is enabled, the gateway controller should delete any existing SQLite DB files at startup before loading artifacts. This enforces the "always starts fresh" guarantee, no residual state from a previous run can influence the current deployment.