Skip to content

Commit 4e3c419

Browse files
Add Install Gateway section in Getting Started guide (#1673)
- Move instructions from the Deploy an Inference Gateway section describing installation of Gateway API CRDs and provider specific GWs Signed-off-by: Dharaneeshwaran Ravichandran <[email protected]>
1 parent 9b6272e commit 4e3c419

File tree

1 file changed

+82
-70
lines changed

1 file changed

+82
-70
lines changed

site-src/guides/index.md

Lines changed: 82 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,76 @@
3737
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/v1.0.2/v1-manifests.yaml
3838
```
3939

40+
### Install the Gateway
41+
42+
Choose one of the following options to install Gateway.
43+
44+
=== "GKE"
45+
46+
Nothing to install here, you can move to the next [section](#deploy-the-inferencepool-and-endpoint-picker-extension)
47+
48+
=== "Istio"
49+
50+
1. Requirements
51+
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
52+
53+
2. Install Istio
54+
55+
```
56+
TAG=$(curl https://storage.googleapis.com/istio-build/dev/1.28-dev)
57+
# on Linux
58+
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-linux-amd64.tar.gz
59+
tar -xvf istioctl-$TAG-linux-amd64.tar.gz
60+
# on macOS
61+
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-osx.tar.gz
62+
tar -xvf istioctl-$TAG-osx.tar.gz
63+
# on Windows
64+
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-win.zip
65+
unzip istioctl-$TAG-win.zip
66+
67+
./istioctl install --set tag=$TAG --set hub=gcr.io/istio-testing --set values.pilot.env.ENABLE_GATEWAY_API_INFERENCE_EXTENSION=true
68+
```
69+
70+
=== "Kgateway"
71+
72+
1. Requirements
73+
74+
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
75+
- [Helm](https://helm.sh/docs/intro/install/) installed.
76+
77+
2. Set the Kgateway version and install the Kgateway CRDs.
78+
79+
```bash
80+
KGTW_VERSION=v2.1.0
81+
helm upgrade -i --create-namespace --namespace kgateway-system --version $KGTW_VERSION kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds
82+
```
83+
84+
3. Install Kgateway
85+
86+
```bash
87+
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true
88+
```
89+
90+
=== "Agentgateway"
91+
92+
1. Requirements
93+
94+
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
95+
- [Helm](https://helm.sh/docs/intro/install/) installed.
96+
97+
2. Set the Kgateway version and install the Kgateway CRDs.
98+
99+
```bash
100+
KGTW_VERSION=v2.1.0
101+
helm upgrade -i --create-namespace --namespace kgateway-system --version $KGTW_VERSION kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds
102+
```
103+
104+
3. Install Kgateway
105+
106+
```bash
107+
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true --set agentgateway.enabled=true
108+
```
109+
40110
### Deploy the InferencePool and Endpoint Picker Extension
41111

42112
Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label `app: vllm-llama3-8b-instruct` and listening on port 8000. The Helm install command automatically installs the endpoint-picker, InferencePool along with provider specific resources.
@@ -59,7 +129,7 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
59129
See [Deploy Inference Gateways](https://cloud.google.com/kubernetes-engine/docs/how-to/deploy-gke-inference-gateway)
60130
for detailed instructions.
61131

62-
2. Deploy Inference Gateway:
132+
2. Deploy the Inference Gateway:
63133

64134
```bash
65135
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/gke/gateway.yaml
@@ -89,34 +159,13 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
89159
Please note that this feature is currently in an experimental phase and is not intended for production use.
90160
The implementation and user experience are subject to changes as we continue to iterate on this project.
91161

92-
1. Requirements
93-
94-
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
95-
96-
2. Install Istio
97-
98-
```
99-
TAG=$(curl https://storage.googleapis.com/istio-build/dev/1.28-dev)
100-
# on Linux
101-
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-linux-amd64.tar.gz
102-
tar -xvf istioctl-$TAG-linux-amd64.tar.gz
103-
# on macOS
104-
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-osx.tar.gz
105-
tar -xvf istioctl-$TAG-osx.tar.gz
106-
# on Windows
107-
wget https://storage.googleapis.com/istio-build/dev/$TAG/istioctl-$TAG-win.zip
108-
unzip istioctl-$TAG-win.zip
109-
110-
./istioctl install --set tag=$TAG --set hub=gcr.io/istio-testing --set values.pilot.env.ENABLE_GATEWAY_API_INFERENCE_EXTENSION=true
111-
```
112-
113-
3. If your EPP uses secure serving with self-signed certs (default), temporarily bypass TLS verification:
162+
1. If your EPP uses secure serving with self-signed certs (default), temporarily bypass TLS verification:
114163

115164
```bash
116165
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/istio/destination-rule.yaml
117166
```
118167

119-
4. Deploy Gateway
168+
2. Deploy the Inference Gateway
120169

121170
```bash
122171
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/istio/gateway.yaml
@@ -129,13 +178,13 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
129178
inference-gateway inference-gateway <MY_ADDRESS> True 22s
130179
```
131180

132-
5. Deploy the HTTPRoute
181+
3. Deploy the HTTPRoute
133182

134183
```bash
135184
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/istio/httproute.yaml
136185
```
137186

138-
6. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
187+
4. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
139188

140189
```bash
141190
kubectl get httproute llm-route -o yaml
@@ -144,28 +193,9 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
144193
=== "Kgateway"
145194

146195
[Kgateway](https://kgateway.dev/) is a Gateway API and Inference Gateway
147-
[conformant](https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/conformance/reports/v1.0.0/gateway/kgateway)
148-
gateway. Follow these steps to run Kgateway:
149-
150-
1. Requirements
151-
152-
- [Helm](https://helm.sh/docs/intro/install/) installed.
153-
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
196+
[conformant](https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/conformance/reports/v1.0.0/gateway/kgateway) gateway. Follow these steps to run Kgateway:
154197

155-
2. Set the Kgateway version and install the Kgateway CRDs.
156-
157-
```bash
158-
KGTW_VERSION=v2.1.0
159-
helm upgrade -i --create-namespace --namespace kgateway-system --version $KGTW_VERSION kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds
160-
```
161-
162-
3. Install Kgateway
163-
164-
```bash
165-
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true
166-
```
167-
168-
4. Deploy the Gateway
198+
1. Deploy the Inference Gateway
169199

170200
```bash
171201
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/kgateway/gateway.yaml
@@ -178,13 +208,13 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
178208
inference-gateway kgateway <MY_ADDRESS> True 22s
179209
```
180210

181-
5. Deploy the HTTPRoute
211+
2. Deploy the HTTPRoute
182212

183213
```bash
184214
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/kgateway/httproute.yaml
185215
```
186216

187-
6. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
217+
3. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
188218

189219
```bash
190220
kubectl get httproute llm-route -o yaml
@@ -196,25 +226,7 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
196226
Agentgateway integrates with [Kgateway](https://kgateway.dev/) as it's control plane. Follow these steps to run Kgateway with the agentgateway
197227
data plane:
198228

199-
1. Requirements
200-
201-
- [Helm](https://helm.sh/docs/intro/install/) installed.
202-
- Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed.
203-
204-
2. Set the Kgateway version and install the Kgateway CRDs.
205-
206-
```bash
207-
KGTW_VERSION=v2.1.0
208-
helm upgrade -i --create-namespace --namespace kgateway-system --version $KGTW_VERSION kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds
209-
```
210-
211-
3. Install Kgateway
212-
213-
```bash
214-
helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true --set agentgateway.enabled=true
215-
```
216-
217-
4. Deploy the Gateway
229+
1. Deploy the Inference Gateway
218230

219231
```bash
220232
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/agentgateway/gateway.yaml
@@ -227,13 +239,13 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
227239
inference-gateway agentgateway <MY_ADDRESS> True 22s
228240
```
229241

230-
5. Deploy the HTTPRoute
242+
2. Deploy the HTTPRoute
231243

232244
```bash
233245
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/agentgateway/httproute.yaml
234246
```
235247

236-
6. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
248+
3. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:
237249

238250
```bash
239251
kubectl get httproute llm-route -o yaml

0 commit comments

Comments
 (0)