Skip to content

Commit c2cea0f

Browse files
committed
fix(self-managed): forward remote Watch discovery
Signed-off-by: Mike Camp <mcamp@nvidia.com>
1 parent d175ccc commit c2cea0f

4 files changed

Lines changed: 43 additions & 0 deletions

File tree

deploy/stacks/self-managed/environments/base.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,12 @@ addons:
314314
# window; Kubernetes cannot mutate one workload kind into the other.
315315
kind: Deployment
316316

317+
discovery:
318+
# Recursive Watch endpoints require an explicit HTTPS URI. Plaintext
319+
# HTTP is available only through the development-only opt-in below.
320+
remoteWatchUrls: []
321+
allowInsecureRemoteWatchHttp: false
322+
317323
# Authority and SNI aware backend router. A worker holds one registration
318324
# stream and one reverse QUIC tunnel per router replica, and each must
319325
# reach the specific replica named by the gRPC authority or QUIC SNI, so

deploy/stacks/self-managed/global.yaml.gotmpl

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -878,6 +878,10 @@ llmRequestRouter:
878878
replicaCount: {{ dig "addons" "llm" "requestRouter" "replicaCount" 3 .Values }}
879879
workload:
880880
kind: {{ dig "addons" "llm" "requestRouter" "workload" "kind" "Deployment" .Values | quote }}
881+
discovery:
882+
remoteWatchUrls:
883+
{{- toYaml (dig "addons" "llm" "requestRouter" "discovery" "remoteWatchUrls" (list) .Values) | nindent 6 }}
884+
allowInsecureRemoteWatchHttp: {{ dig "addons" "llm" "requestRouter" "discovery" "allowInsecureRemoteWatchHttp" false .Values }}
881885
service:
882886
grpcPort: {{ $llmRequestRouterGrpcPort }}
883887
{{- if .Values.global.imagePullSecrets }}

deploy/stacks/self-managed/tests/llm-router-local-chart.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ test "$actual" = "$chart_path" || {
3131
--environment default \
3232
--state-values-set addons.llm.enabled=true \
3333
--state-values-set-string "addons.llm.requestRouter.chartPath=$chart_path" \
34+
--state-values-set-string 'addons.llm.requestRouter.discovery.remoteWatchUrls[0]=https://region-b.example.invalid:50071' \
35+
--state-values-set addons.llm.requestRouter.discovery.allowInsecureRemoteWatchHttp=true \
3436
--state-values-set ingress.gatewayApi.gateways.shared.name=shared-gw \
3537
--state-values-set ingress.gatewayApi.gateways.shared.namespace=envoy-gateway-system \
3638
--state-values-set ingress.gatewayApi.gateways.grpc.name=grpc-gw \
@@ -51,6 +53,18 @@ test "$default_workload_kind" = "Deployment" || {
5153
exit 1
5254
}
5355

56+
remote_watch_url="$(yq -r '.llmRequestRouter.discovery.remoteWatchUrls[0]' "$values_file")"
57+
test "$remote_watch_url" = "https://region-b.example.invalid:50071" || {
58+
echo "llm-router-local-chart: expected remote Watch URL forwarding, got ${remote_watch_url:-missing}" >&2
59+
exit 1
60+
}
61+
62+
allow_insecure_remote_watch_http="$(yq -r '.llmRequestRouter.discovery.allowInsecureRemoteWatchHttp' "$values_file")"
63+
test "$allow_insecure_remote_watch_http" = "true" || {
64+
echo "llm-router-local-chart: expected development HTTP opt-in forwarding, got ${allow_insecure_remote_watch_http:-missing}" >&2
65+
exit 1
66+
}
67+
5468
(cd "$stack_dir" && HELMFILE_ENV=base helmfile \
5569
--file helmfile.d/02-core.yaml.gotmpl \
5670
--environment default \

docs/user/llm-function-enablement.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,25 @@ both to use the in-cluster backend-router Service. Helmfile rendering rejects a
243243
partial override. The gRPC worker address normally uses the same TCP endpoint
244244
as `pylonGrpcDialAddress`.
245245

246+
To recursively discover request routers in another region, set explicit remote
247+
Watch dial URIs on the self-managed operator surface:
248+
249+
```yaml
250+
addons:
251+
llm:
252+
requestRouter:
253+
discovery:
254+
remoteWatchUrls:
255+
- https://region-b-watch.example.com:50071
256+
```
257+
258+
Each URI must use `https://`. Development-only plaintext endpoints require an
259+
explicit `http://` URI and
260+
`addons.llm.requestRouter.discovery.allowInsecureRemoteWatchHttp: true`.
261+
Scheme-less and unsupported endpoints are rejected rather than defaulting to
262+
plaintext. The dial hostname selects TLS SNI; identities advertised by the
263+
remote Watch response remain the HTTP/2 authorities used for registration.
264+
246265
The scheme-less `global.workerEndpoints.llmRequestRouterAddress` in this
247266
example is the self-managed profile's initial `host:port` input. The
248267
`pylonGrpcDialAddress` override requires an explicit `http://` or `https://`

0 commit comments

Comments
 (0)