Skip to content

Commit 3650838

Browse files
danehanssallyom
authored andcommitted
Docs: Updates Latest/Main Quickstart (kubernetes-sigs#1747)
Signed-off-by: Daneyon Hansen <[email protected]>
1 parent 5cd334a commit 3650838

File tree

2 files changed

+53
-7
lines changed

2 files changed

+53
-7
lines changed

site-src/_includes/epp-latest.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
=== "GKE"
2+
3+
```bash
4+
export GATEWAY_PROVIDER=gke
5+
helm install vllm-llama3-8b-instruct \
6+
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
7+
--set provider.name=$GATEWAY_PROVIDER \
8+
--version $IGW_CHART_VERSION \
9+
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
10+
```
11+
12+
=== "Istio"
13+
14+
```bash
15+
export GATEWAY_PROVIDER=istio
16+
helm install vllm-llama3-8b-instruct \
17+
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
18+
--set provider.name=$GATEWAY_PROVIDER \
19+
--version $IGW_CHART_VERSION \
20+
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
21+
```
22+
23+
=== "Kgateway"
24+
25+
```bash
26+
export GATEWAY_PROVIDER=none
27+
helm install vllm-llama3-8b-instruct \
28+
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
29+
--set provider.name=$GATEWAY_PROVIDER \
30+
--version $IGW_CHART_VERSION \
31+
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
32+
```
33+
34+
=== "Agentgateway"
35+
36+
```bash
37+
export GATEWAY_PROVIDER=none
38+
helm install vllm-llama3-8b-instruct \
39+
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
40+
--set provider.name=$GATEWAY_PROVIDER \
41+
--version $IGW_CHART_VERSION \
42+
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
43+
```

site-src/guides/getting-started-latest.md

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -26,13 +26,13 @@
2626
--8<-- "site-src/_includes/model-server-cpu.md"
2727

2828
```bash
29-
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/vllm/sim-deployment.yaml
29+
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/vllm/cpu-deployment.yaml
3030
```
3131

3232
--8<-- "site-src/_includes/model-server-sim.md"
3333

3434
```bash
35-
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.0/config/manifests/vllm/sim-deployment.yaml
35+
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/vllm/sim-deployment.yaml
3636
```
3737

3838
### Install the Inference Extension CRDs
@@ -51,7 +51,7 @@ kubectl apply -k https://github.com/kubernetes-sigs/gateway-api-inference-extens
5151
export IGW_CHART_VERSION=v0
5252
```
5353

54-
--8<-- "site-src/_includes/epp.md"
54+
--8<-- "site-src/_includes/epp-latest.md"
5555

5656
### Deploy an Inference Gateway
5757

@@ -147,8 +147,9 @@ kubectl apply -k https://github.com/kubernetes-sigs/gateway-api-inference-extens
147147

148148
=== "Kgateway"
149149

150-
[Kgateway](https://kgateway.dev/) added Inference Gateway support as a **technical preview** in the
151-
[v2.0.0 release](https://github.com/kgateway-dev/kgateway/releases/tag/v2.0.0). InferencePool v1.0.1 is currently supported in the latest [rolling release](https://github.com/kgateway-dev/kgateway/releases/tag/v2.1.0-main), which includes the latest changes but may be unstable until the [v2.1.0 release](https://github.com/kgateway-dev/kgateway/milestone/58) is published.
150+
[Kgateway](https://kgateway.dev/) is a Gateway API and Inference Gateway
151+
[conformant](https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/conformance/reports/v1.0.0/gateway/kgateway)
152+
gateway. Follow these steps to run Kgateway:
152153

153154
1. Requirements
154155

@@ -195,7 +196,9 @@ kubectl apply -k https://github.com/kubernetes-sigs/gateway-api-inference-extens
195196

196197
=== "Agentgateway"
197198

198-
[Agentgateway](https://agentgateway.dev/) is a purpose-built proxy designed for AI workloads, and comes with native support for Inference Gateway. Agentgateway integrates with [Kgateway](https://kgateway.dev/) as it's control plane. InferencePool v1.0.0 is currently supported in the latest [rolling release](https://github.com/kgateway-dev/kgateway/releases/tag/v2.1.0-main), which includes the latest changes but may be unstable until the [v2.1.0 release](https://github.com/kgateway-dev/kgateway/milestone/58) is published.
199+
[Agentgateway](https://agentgateway.dev/) is a purpose-built proxy designed for AI workloads, and comes with native support for Inference Gateway.
200+
Agentgateway integrates with [Kgateway](https://kgateway.dev/) as it's control plane. Follow these steps to run Kgateway with the agentgateway
201+
data plane:
199202

200203
1. Requirements
201204

@@ -212,7 +215,7 @@ kubectl apply -k https://github.com/kubernetes-sigs/gateway-api-inference-extens
212215
3. Install Kgateway
213216

214217
```bash
215-
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true --set agentGateway.enabled=true
218+
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true --set agentgateway.enabled=true
216219
```
217220

218221
4. Deploy the Gateway

0 commit comments

Comments
 (0)