Skip to content

Commit ed1af5d

Browse files
authored
Update multiple pool guide for agentgateway (kubernetes-sigs#2896)
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
1 parent c489161 commit ed1af5d

1 file changed

Lines changed: 11 additions & 15 deletions

File tree

site-src/guides/serving-multiple-inference-pools-latest.md

Lines changed: 11 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -43,13 +43,14 @@ The BBR extracts the model name from the request body, does a lookup of the base
4343
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/body-based-routing
4444
```
4545

46-
=== "Kgateway"
46+
=== "Agentgateway"
4747

48-
Kgateway does not require the Body-Based Routing Extension, and instead natively implements Body-Based Routing.
48+
Agentgateway does not require the Body-Based Routing Extension, and instead natively implements Body-Based Routing.
4949
To use Body Based Routing, apply an `AgentgatewayPolicy`:
5050

51-
```yaml
52-
apiVersion: gateway.kgateway.dev/v1alpha1
51+
```bash
52+
kubectl apply -f- <<EOF
53+
apiVersion: agentgateway.dev/v1alpha1
5354
kind: AgentgatewayPolicy
5455
metadata:
5556
name: bbr
@@ -65,13 +66,8 @@ The BBR extracts the model name from the request body, does a lookup of the base
6566
set:
6667
- name: X-Gateway-Base-Model-Name
6768
value: |
68-
{
69-
"Qwen/Qwen3-32B": "Qwen/Qwen3-32B",
70-
"food-review-1": "Qwen/Qwen3-32B",
71-
"deepseek/DeepSeek-r1": "deepseek/DeepSeek-r1",
72-
"ski-resorts": "deepseek/DeepSeek-r1",
73-
"movie-critique": "deepseek/DeepSeek-r1",
74-
}[json(request.body).model]
69+
{"Qwen/Qwen3-32B": "Qwen/Qwen3-32B", "food-review-1": "Qwen/Qwen3-32B", "deepseek/DeepSeek-r1": "deepseek/DeepSeek-r1", "ski-resorts": "deepseek/DeepSeek-r1", "movie-critique": "deepseek/DeepSeek-r1"}[string(json(request.body).model)]
70+
EOF
7571
```
7672

7773
=== "Other"
@@ -132,7 +128,7 @@ Set the Helm chart version (unless already set).
132128
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
133129
```
134130

135-
=== "Kgateway"
131+
=== "Agentgateway"
136132

137133
```bash
138134
export GATEWAY_PROVIDER=none
@@ -141,7 +137,7 @@ Set the Helm chart version (unless already set).
141137
--set inferencePool.modelServers.matchLabels.app=vllm-deepseek-r1 \
142138
--set provider.name=$GATEWAY_PROVIDER \
143139
--set experimentalHttpRoute.enabled=true \
144-
--set experimentalHttpRoute.baseModel=deepseek/vllm-deepseek-r1 \
140+
--set experimentalHttpRoute.baseModel=deepseek/DeepSeek-r1 \
145141
--version $IGW_CHART_VERSION \
146142
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
147143
```
@@ -153,7 +149,7 @@ Set the Helm chart version (unless already set).
153149
--dependency-update \
154150
--set inferencePool.modelServers.matchLabels.app=vllm-deepseek-r1 \
155151
--set experimentalHttpRoute.enabled=true \
156-
--set experimentalHttpRoute.baseModel=deepseek/vllm-deepseek-r1 \
152+
--set experimentalHttpRoute.baseModel=deepseek/DeepSeek-r1 \
157153
--version $IGW_CHART_VERSION \
158154
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
159155
```
@@ -204,7 +200,7 @@ Run `helm upgrade` in order to update in place the HttpRoute mapping of the firs
204200
--version $IGW_CHART_VERSION
205201
```
206202

207-
=== "Kgateway"
203+
=== "Agentgateway"
208204

209205
```bash
210206
export GATEWAY_PROVIDER=none

0 commit comments

Comments
 (0)