feat(llm-request-router): support Deployment workloads

deploy/helm/llm-request-router/Makefile

@@ -71,10 +71,10 @@ check-pki-render:
 	sh ./scripts/check-pki-render.sh
 
 check-multi-replica-render:
-	sh ./scripts/check-multi-replica-render.sh
+	bash ./scripts/check-multi-replica-render.sh
 
 check-backend-router-render:
-	sh ./scripts/check-backend-router-render.sh
+	bash ./scripts/check-backend-router-render.sh
 
 clean:
 	rm -rf ./packaged-charts bin

deploy/helm/llm-request-router/README.md

@@ -4,12 +4,12 @@ This repository contains the Helm chart for deploying the NVCF LLM Request Route
 
 ## Overview
 
-The chart packages the LLM Request Router StatefulSet with HTTP and gRPC
-services, a metrics endpoint, and a headless service for multi-instance DNS
-discovery. It can also deploy the Stargate Kubernetes backend router for
-worker gRPC registration and reverse QUIC tunnels through a shared Gateway or
-load balancer. The backend router selects the correct Stargate pod from gRPC
-authority and QUIC SNI.
+The chart packages the LLM Request Router as either a Deployment or a
+StatefulSet, with Deployment as the default. It includes HTTP and gRPC
+services, a metrics endpoint, and a headless service. It can also deploy the
+Stargate Kubernetes backend router for worker gRPC registration and reverse
+QUIC tunnels through a shared Gateway or load balancer. The backend router
+selects the correct Stargate pod from gRPC authority and QUIC SNI.
 
 A Vault Agent sidecar is configured to fetch a service token from a Vault or
 OpenBao backend. The application reads `nvcfApiToken` from
@@ -28,11 +28,37 @@ llmRequestRouter:
     tag: <appVersion>
 ```
 
-Single-replica deployments may use self-only discovery with `llmRequestRouter.discovery.disableDnsDiscovery=true`. Multi-replica deployments require DNS discovery and stable per-pod identity, so the chart fails rendering if DNS discovery is disabled while `llmRequestRouter.replicaCount > 1`. For multi-replica deployments, the default advertised hostname template is `{pod_name}.<headless-service>.<namespace>.svc.cluster.local`; the StatefulSet and headless service provide the stable pod DNS names required for router replicas to discover each other and share backend registrations.
-
-`llmRequestRouter.kubernetes.advertisedHostnameTemplate` supports the Stargate placeholders `{pod_name}` and `{namespace}`. Stargate resolves both placeholders at runtime. For certificate validation, the chart substitutes the deployment namespace and a representative StatefulSet pod name. `{pod_name}` must stay within the leftmost DNS label when certificate coverage relies on a wildcard. When `llmRequestRouter.certificate.enabled=true`, `certificate.dnsNames` must cover the advertised hostname with either a case-insensitive exact name or a valid leftmost `*.` wildcard. A wildcard covers exactly one label and requires at least two suffix labels. For example, `*.nvcf.example.internal` covers `{pod_name}.nvcf.example.internal`, but `*.example.internal` does not cover `{pod_name}.nvcf.example.internal`.
-
-Upgrading from a chart version that rendered a Deployment can briefly run both the old Deployment and new StatefulSet during `helm upgrade` while Helm replaces the workload kind.
+Single-replica Deployments automatically use self-only discovery so their
+headless Service cannot introduce dashed-IP SRV aliases. A multi-replica
+Deployment requires the backend router. The default `null` value for
+`llmRequestRouter.backendRouter.enabled` enables it automatically in that
+topology; explicitly setting `false` is rejected. The backend router builds
+Watch responses and forwarding routes from the same
+EndpointSlice snapshot. Each ready endpoint is keyed by its Pod
+`targetRef.name`, so one pod produces one canonical identity even when DNS also
+exposes a dashed-IP SRV alias. A multi-replica StatefulSet can instead run
+without the backend router and retain direct headless Service SRV discovery.
+`llmRequestRouter.discovery.watchHeartbeatMs` controls the maximum interval
+between unchanged Watch snapshots from both Stargate and the backend router.
+
+`llmRequestRouter.kubernetes.advertisedHostnameTemplate` supports the Stargate
+placeholders `{pod_name}` and `{namespace}`. Stargate resolves both placeholders
+at runtime. For certificate validation, the chart substitutes the deployment
+namespace and a representative pod name. `{pod_name}` must stay within the
+leftmost DNS label when certificate coverage relies on a wildcard. When
+`llmRequestRouter.certificate.enabled=true`, `certificate.dnsNames` must cover
+the advertised hostname with either a case-insensitive exact name or a valid
+leftmost `*.` wildcard. A wildcard covers exactly one label and requires at
+least two suffix labels. For example, `*.nvcf.example.internal` covers
+`{pod_name}.nvcf.example.internal`, but `*.example.internal` does not cover
+`{pod_name}.nvcf.example.internal`.
+
+Existing installations that currently run the StatefulSet must set
+`llmRequestRouter.workload.kind=StatefulSet` before upgrading to this chart.
+Changing `workload.kind` is a controlled migration, not an in-place Kubernetes
+mutation. Plan a maintenance window, remove or rename the old workload, and
+verify that only the selected kind owns the request-router Pods before scaling
+it. A plain Helm upgrade across workload kinds can briefly run both workloads.
 
 ## Prerequisites
 
@@ -80,6 +106,8 @@ Important settings to review before deployment:
 
 - `llmRequestRouter.image.*` for the router container image
 - `llmRequestRouter.imagePullSecrets` for private registry access
+- `llmRequestRouter.workload.kind` to select `Deployment` (default) or `StatefulSet`
+- `llmRequestRouter.workload.deployment.strategy` and `llmRequestRouter.workload.statefulSet.*` for workload-specific rollout settings
 - `llmRequestRouter.replicaCount`, resource requests, and limits for your environment
 - `llmRequestRouter.service.*` for HTTP, gRPC, metrics, and headless service ports
 - `llmRequestRouter.backendRouter.*` for multi-replica worker gRPC and reverse-tunnel routing
@@ -96,9 +124,10 @@ The default values include development-oriented placeholders. Override them befo
 
 ## Backend Worker Routing
 
-Enable `llmRequestRouter.backendRouter.enabled` when workers reach a
-multi-replica request router through a shared endpoint. Set both pylon dial
-addresses to the external endpoints that workers can resolve:
+The backend router is enabled automatically for a multi-replica Deployment.
+Set `llmRequestRouter.backendRouter.enabled=true` explicitly when workers reach
+another supported workload topology through a shared endpoint. Set both pylon
+dial addresses to the external endpoints that workers can resolve:
 
 ```yaml
 llmRequestRouter:
@@ -114,8 +143,11 @@ chart appVersion. That image must contain
 `/usr/local/bin/stargate-k8s-router`; override `backendRouter.image.*` only to
 validate a different Stargate build.
 
-The backend router watches EndpointSlices. The chart creates a dedicated
-ServiceAccount by default and binds a namespaced Role to it when
+The backend router watches EndpointSlices and publishes those ready targets
+directly through `WatchStargates`. It uses the same snapshot for gRPC and QUIC
+forwarding, so a removed or replaced Pod cannot remain as a discovery-only
+target. The chart creates a dedicated ServiceAccount by default and binds a
+namespaced Role to it when
 `llmRequestRouter.rbac.create=true`. When
 `llmRequestRouter.backendRouter.serviceAccount.create=false`, set
 `llmRequestRouter.backendRouter.serviceAccount.name` to an existing account.

deploy/helm/llm-request-router/llm-request-router/templates/_helpers.tpl

@@ -76,6 +76,30 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
 {{- default .Release.Namespace .Values.llmRequestRouter.namespace -}}
 {{- end -}}
 
+{{- define "llm-request-router.workloadKind" -}}
+{{- dig "workload" "kind" "Deployment" .Values.llmRequestRouter | toString -}}
+{{- end -}}
+
+{{/*
+An unset backendRouter.enabled follows the workload contract: a multi-replica
+Deployment needs the EndpointSlice router, while StatefulSet and single-replica
+direct modes retain their previous behavior. An explicit boolean always wins;
+deployment.yaml rejects the unsafe explicit-false combination.
+*/}}
+{{- define "llm-request-router.backendRouterEnabled" -}}
+{{- $backendRouter := .Values.llmRequestRouter.backendRouter | default dict -}}
+{{- $configured := get $backendRouter "enabled" -}}
+{{- if kindIs "bool" $configured -}}
+{{- $configured -}}
+{{- else if and
+      (eq (include "llm-request-router.workloadKind" .) "Deployment")
+      (gt (.Values.llmRequestRouter.replicaCount | int) 1) -}}
+true
+{{- else -}}
+false
+{{- end -}}
+{{- end -}}
+
 {{- define "llm-request-router.isValidDnsName" -}}
 {{- $name := .name | toString | lower -}}
 {{- $labels := splitList "." $name -}}
@@ -103,7 +127,7 @@ rendering must agree on that list, so both read it from here.
 {{- define "llm-request-router.effectiveCertificateDnsNames" -}}
 {{- $certificate := .Values.llmRequestRouter.certificate | default dict -}}
 {{- $dnsNames := dig "dnsNames" (list) $certificate -}}
-{{- if dig "backendRouter" "enabled" false .Values.llmRequestRouter -}}
+{{- if eq (include "llm-request-router.backendRouterEnabled" .) "true" -}}
 {{- $wildcard := replace "{pod_name}" "*" (include "llm-request-router.advertisedHostnameTemplate" .) -}}
 {{- if not (has $wildcard $dnsNames) -}}
 {{- $dnsNames = append $dnsNames $wildcard -}}
@@ -250,7 +274,7 @@ Coverage, matching the role flags:
 
 {{- define "llm-request-router.advertisedHostnameTemplate" -}}
 {{- $configured := .Values.llmRequestRouter.kubernetes.advertisedHostnameTemplate -}}
-{{- $backendRouterEnabled := dig "backendRouter" "enabled" false .Values.llmRequestRouter -}}
+{{- $backendRouterEnabled := eq (include "llm-request-router.backendRouterEnabled" .) "true" -}}
 {{- if and $backendRouterEnabled $configured (ne (len (splitList "{pod_name}" $configured)) 2) -}}
 {{- fail "llmRequestRouter.kubernetes.advertisedHostnameTemplate must contain exactly one {pod_name} when backendRouter.enabled is true" -}}
 {{- end -}}

deploy/helm/llm-request-router/llm-request-router/templates/backend-router-poddisruptionbudget.yaml

@@ -3,7 +3,7 @@
 
 {{- $backendRouter := .Values.llmRequestRouter.backendRouter -}}
 {{- $pdb := $backendRouter.podDisruptionBudget | default dict -}}
-{{- if and $backendRouter.enabled $pdb.enabled }}
+{{- if and (eq (include "llm-request-router.backendRouterEnabled" .) "true") $pdb.enabled }}
 {{- $pdbMinAvailable := $pdb.minAvailable | toString -}}
 {{- $pdbMaxUnavailable := $pdb.maxUnavailable | toString -}}
 {{- if and (ne $pdbMinAvailable "") (ne $pdbMaxUnavailable "") }}

deploy/helm/llm-request-router/llm-request-router/templates/backend-router-rbac.yaml

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
-{{- if and .Values.llmRequestRouter.backendRouter.enabled .Values.llmRequestRouter.rbac.create }}
+{{- if and (eq (include "llm-request-router.backendRouterEnabled" .) "true") .Values.llmRequestRouter.rbac.create }}
 apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
 metadata:

deploy/helm/llm-request-router/llm-request-router/templates/backend-router-serviceaccount.yaml

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
-{{- if and .Values.llmRequestRouter.backendRouter.enabled .Values.llmRequestRouter.backendRouter.serviceAccount.create }}
+{{- if and (eq (include "llm-request-router.backendRouterEnabled" .) "true") .Values.llmRequestRouter.backendRouter.serviceAccount.create }}
 apiVersion: v1
 kind: ServiceAccount
 metadata:

deploy/helm/llm-request-router/llm-request-router/templates/backend-router-servicemonitor.yaml

@@ -3,7 +3,7 @@
 
 {{- $metricsEnabled := dig "metrics" "enabled" false .Values.llmRequestRouter }}
 {{- $serviceMonitorEnabled := dig "metrics" "serviceMonitor" "enabled" false .Values.llmRequestRouter }}
-{{- if and .Values.llmRequestRouter.backendRouter.enabled $metricsEnabled $serviceMonitorEnabled }}
+{{- if and (eq (include "llm-request-router.backendRouterEnabled" .) "true") $metricsEnabled $serviceMonitorEnabled }}
 apiVersion: monitoring.coreos.com/v1
 kind: ServiceMonitor
 metadata:

deploy/helm/llm-request-router/llm-request-router/templates/backend-router.yaml

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
-{{- if .Values.llmRequestRouter.backendRouter.enabled }}
+{{- if eq (include "llm-request-router.backendRouterEnabled" .) "true" }}
 {{- include "llm-request-router.validateBackendRouterTls" . }}
 {{- include "llm-request-router.validateBackendRouterServiceAccount" . }}
 {{- $advertisedHostnameTemplate := include "llm-request-router.advertisedHostnameTemplate" . }}
@@ -81,6 +81,9 @@ spec:
             - --target-namespace={{ include "llm-request-router.namespace" . }}
             - --target-service-name={{ include "llm-request-router.fullname" . }}
             - --advertised-hostname-template={{ $advertisedHostnameTemplate }}
+            - --advertised-grpc-port={{ .Values.llmRequestRouter.service.grpcPort }}
+            - --grpc-pylon-dial-addr={{ include "llm-request-router.backendRouterGrpcDialAddress" . }}
+            - --watch-heartbeat-ms={{ .Values.llmRequestRouter.discovery.watchHeartbeatMs }}
             - --grpc-port-name=grpc
             - --quic-port-name=quic
             - --tunnel-protocol=raw-quic

deploy/helm/llm-request-router/llm-request-router/templates/deployment.yaml

@@ -13,22 +13,37 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+{{- $workloadKind := include "llm-request-router.workloadKind" . -}}
+{{- if not (has $workloadKind (list "Deployment" "StatefulSet")) -}}
+{{- fail (printf "llmRequestRouter.workload.kind must be Deployment or StatefulSet, got %q" $workloadKind) -}}
+{{- end }}
 apiVersion: apps/v1
-kind: StatefulSet
+kind: {{ $workloadKind }}
 metadata:
   name: {{ include "llm-request-router.fullname" . }}
   namespace: {{ include "llm-request-router.namespace" . }}
   labels:
     {{- include "llm-request-router.labels" . | nindent 4 }}
 {{- /*
-Single-replica deployments can advertise the stable service DNS because it maps
-to one router pod. Multi-replica deployments need per-pod advertised hostnames
-so reverse-tunnel clients connect back to the router instance that issued the
-tunnel target.
+Single-replica Deployments use only their local identity because Kubernetes
+may publish both pod-name and dashed-IP SRV records for Deployment endpoints.
+StatefulSets retain stable DNS identities. Multi-replica Deployments need the
+EndpointSlice backend router so reverse-tunnel clients connect back to the
+router instance that issued the tunnel target.
 */}}
 {{- $replicaCount := .Values.llmRequestRouter.replicaCount | int }}
-{{- $disableDnsDiscovery := dig "discovery" "disableDnsDiscovery" false .Values.llmRequestRouter }}
-{{- if and $disableDnsDiscovery (gt $replicaCount 1) }}
+{{- $backendRouterEnabled := eq (include "llm-request-router.backendRouterEnabled" .) "true" }}
+{{- $configuredDisableDnsDiscovery := dig "discovery" "disableDnsDiscovery" false .Values.llmRequestRouter }}
+{{- $directDeployment := and (eq $workloadKind "Deployment") (not $backendRouterEnabled) }}
+{{- $disableDnsDiscovery := or $configuredDisableDnsDiscovery $directDeployment }}
+{{- $watchHeartbeatMs := .Values.llmRequestRouter.discovery.watchHeartbeatMs | int }}
+{{- if le $watchHeartbeatMs 0 }}
+{{- fail "llmRequestRouter.discovery.watchHeartbeatMs must be greater than 0" }}
+{{- end }}
+{{- if and (eq $workloadKind "Deployment") (gt $replicaCount 1) (not $backendRouterEnabled) }}
+{{- fail "llmRequestRouter.backendRouter.enabled must be true when llmRequestRouter.workload.kind is Deployment and replicaCount is greater than 1" }}
+{{- end }}
+{{- if and $configuredDisableDnsDiscovery (gt $replicaCount 1) (not $backendRouterEnabled) }}
 {{- fail "llmRequestRouter.discovery.disableDnsDiscovery cannot be true when llmRequestRouter.replicaCount is greater than 1; multi-replica routers require DNS discovery" }}
 {{- end }}
 {{- /*
@@ -37,7 +52,6 @@ renders no Certificate, so a guard in that template would never run.
 */}}
 {{- include "llm-request-router.validateTlsIdentity" . }}
 {{- $advertisedHostnameTemplate := include "llm-request-router.advertisedHostnameTemplate" . }}
-{{- $backendRouterEnabled := dig "backendRouter" "enabled" false .Values.llmRequestRouter }}
 {{- if and $backendRouterEnabled (not .Values.llmRequestRouter.transport.reverseTunnelListenAddr) }}
 {{- fail "llmRequestRouter.backendRouter.enabled requires llmRequestRouter.transport.reverseTunnelListenAddr" }}
 {{- end }}
@@ -54,9 +68,16 @@ renders no Certificate, so a guard in that template would never run.
 {{- end }}
 {{- end }}
 spec:
+  {{- if eq $workloadKind "Deployment" }}
+  strategy:
+    {{- toYaml .Values.llmRequestRouter.workload.deployment.strategy | nindent 4 }}
+  {{- else }}
   serviceName: {{ .Values.llmRequestRouter.service.headlessName }}
+  podManagementPolicy: {{ .Values.llmRequestRouter.workload.statefulSet.podManagementPolicy }}
+  updateStrategy:
+    {{- toYaml .Values.llmRequestRouter.workload.statefulSet.updateStrategy | nindent 4 }}
+  {{- end }}
   replicas: {{ $replicaCount }}
-  podManagementPolicy: Parallel
   selector:
     matchLabels:
       {{- include "llm-request-router.selectorLabels" . | nindent 6 }}

deploy/helm/llm-request-router/llm-request-router/values.yaml

@@ -15,10 +15,24 @@
 
 llmRequestRouter:
   enabled: true
-  # Multi-replica requires DNS discovery and stable per-pod hostnames from
-  # the StatefulSet and headless service.
   replicaCount: 3
 
+  workload:
+    # Deployment matches managed NVCF. Existing installations can pin
+    # StatefulSet and migrate workload kinds in a controlled maintenance
+    # window; Kubernetes cannot mutate one workload kind into the other.
+    kind: Deployment
+    deployment:
+      strategy:
+        type: RollingUpdate
+        rollingUpdate:
+          maxSurge: 1
+          maxUnavailable: 0
+    statefulSet:
+      podManagementPolicy: Parallel
+      updateStrategy:
+        type: RollingUpdate
+
   podDisruptionBudget:
     enabled: false
     # minAvailable and maxUnavailable are mutually exclusive; set exactly one.
@@ -72,11 +86,14 @@ llmRequestRouter:
     headlessName: llm-request-router-headless
 
   # Routes backend registration and reverse-tunnel traffic to the Stargate pod
-  # named by gRPC authority or QUIC SNI. Enable this when pylons connect through
-  # a shared Gateway or load balancer and request-router replicaCount is greater
-  # than one.
+  # named by gRPC authority or QUIC SNI. Multi-replica Deployments enable this
+  # automatically. Other workload topologies can enable it when pylons connect
+  # through a shared Gateway or load balancer.
   backendRouter:
-    enabled: false
+    # null enables the backend router automatically for a multi-replica
+    # Deployment and disables it otherwise. Set a boolean to override this;
+    # false is rejected for a multi-replica Deployment.
+    enabled: null
     # The router is the single front door for worker registration and reverse
     # tunnels in its region, so one replica makes it a single point of failure
     # for every worker in that region. The router keeps no state beyond the QUIC
@@ -218,6 +235,9 @@ llmRequestRouter:
 
   discovery:
     disableDnsDiscovery: false
+    # Maximum interval between unchanged WatchStargates snapshots. This is
+    # shared by Stargate and the EndpointSlice backend router.
+    watchHeartbeatMs: 5000
 
   transport:
     quicConnectTimeoutMs: 2000