Skip to content

chore: support hostname in aigateway route#1987

Open
xianml wants to merge 1 commit intoenvoyproxy:mainfrom
bentoml:chore/gateway-hostname-cr
Open

chore: support hostname in aigateway route#1987
xianml wants to merge 1 commit intoenvoyproxy:mainfrom
bentoml:chore/gateway-hostname-cr

Conversation

@xianml
Copy link
Copy Markdown

@xianml xianml commented Mar 25, 2026

Description

Currently, aigetewayroute seems act as cluster level CR, since it did not have a hostname attached to it. It means that we can not specific a group of models under a specific Hostname (say xx.api.aieg.com). And meanwhile, /v1/models will return all the openai models of all aigetewayroute CRs in the cluster.

Related Issues/PRs (if applicable)

Special notes for reviewers (if applicable)

mentioned in #1646

@xianml xianml requested a review from a team as a code owner March 25, 2026 10:27
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 25, 2026
@dosubot
Copy link
Copy Markdown

dosubot bot commented Mar 25, 2026

Related Documentation

3 document(s) may need updating based on files changed in this PR:

Envoy's Space

aigatewayroute-inferencepool /ai-gateway/blob/main/site/docs/capabilities/inference/aigatewayroute-inferencepool.md
View Suggested Changes
@@ -416,6 +416,10 @@
     - name: inference-pool-with-aigwroute
       kind: Gateway
       group: gateway.networking.k8s.io
+  # Optional: Define hostnames to match against the HTTP Host header
+  # hostnames:
+  #   - "api.example.com"
+  #   - "*.myapp.com"
   rules:
     # Route for vLLM Llama model via InferencePool
     - matches:
@@ -507,6 +511,44 @@
 
 ## Advanced Features
 
+### Hostname-Based Routing
+
+AIGatewayRoute supports hostname-based filtering through the `hostnames` field:
+
+- **Optional Field**: Defines a set of hostnames to match against the HTTP Host header
+- **Multiple Hostnames**: Supports up to 16 hostnames per route
+- **Precise Matching**: Match exact hostnames (e.g., `"api.example.com"`)
+- **Wildcard Matching**: Match subdomains with wildcards (e.g., `"*.example.com"`)
+- **Gateway API Compatibility**: Maps directly to HTTPRoute.Spec.Hostnames
+
+When the `hostnames` field is configured, only models from AIGatewayRoutes with matching hostnames are returned by the `/v1/models` endpoint. This enables multi-tenant scenarios where different hostnames expose different model catalogs.
+
+**Example with hostnames:**
+
+```yaml
+apiVersion: aigateway.envoyproxy.io/v1alpha1
+kind: AIGatewayRoute
+metadata:
+  name: example-route
+spec:
+  hostnames:
+    - "api.example.com"
+    - "*.myapp.com"
+  parentRefs:
+    - name: my-gateway
+      kind: Gateway
+  rules:
+    - matches:
+        - headers:
+            - type: Exact
+              name: x-ai-eg-model
+              value: my-model
+      backendRefs:
+        - group: inference.networking.k8s.io
+          kind: InferencePool
+          name: my-pool
+```
+
 ### Model-Based Routing
 
 AIGatewayRoute automatically extracts the model name from the request body and routes to the appropriate backend:

[Accept] [Decline]

connect-providers /ai-gateway/blob/main/site/docs/capabilities/llm-integrations/connect-providers.md
View Suggested Changes
@@ -237,6 +237,7 @@
 #### Purpose and Configuration
 
 - **Request Routing**: Directs traffic to specific backends based on model names or other criteria
+- **Hostname-Based Filtering**: Optionally matches traffic based on HTTP Host header
 - **API Unification**: Provides a consistent interface regardless of backend provider
 - **Request Transformation**: Automatically converts between different API schemas
 - **Load Balancing**: Distributes traffic across multiple backends
@@ -255,6 +256,11 @@
       kind: Gateway
       group: gateway.networking.k8s.io
 
+  # Optional: Filter traffic by hostname
+  hostnames:
+    - "api.example.com"
+    - "*.example.com"
+
   # Routing rules
   rules:
     - matches:
@@ -270,6 +276,51 @@
               name: x-ai-eg-model
               value: claude-3-sonnet
       backendRefs:
+        - name: bedrock-backend
+```
+
+#### Hostname-Based Routing
+
+The optional `hostnames` field allows filtering traffic based on the HTTP Host header before applying routing rules. This field maps directly to HTTPRoute.Spec.Hostnames from the Gateway API.
+
+**Key characteristics:**
+
+- Supports up to 16 hostnames per route
+- Accepts precise hostnames (e.g., `"api.example.com"`) or wildcard patterns (e.g., `"*.example.com"`)
+- Wildcards match subdomains only when there's a label boundary (e.g., `*.example.com` matches `api.example.com` but not `example.com`)
+- Affects the `/v1/models` endpoint: only models from AIGatewayRoutes with matching hostnames are included in the response
+
+**Example with hostname-based routing:**
+
+```yaml
+apiVersion: aigateway.envoyproxy.io/v1beta1
+kind: AIGatewayRoute
+metadata:
+  name: api-subdomain-route
+spec:
+  parentRefs:
+    - name: my-gateway
+      kind: Gateway
+      group: gateway.networking.k8s.io
+  hostnames:
+    - "api.example.com"
+  rules:
+    - backendRefs:
+        - name: openai-backend
+---
+apiVersion: aigateway.envoyproxy.io/v1beta1
+kind: AIGatewayRoute
+metadata:
+  name: wildcard-subdomain-route
+spec:
+  parentRefs:
+    - name: my-gateway
+      kind: Gateway
+      group: gateway.networking.k8s.io
+  hostnames:
+    - "*.internal.example.com"
+  rules:
+    - backendRefs:
         - name: bedrock-backend
 ```
 

[Accept] [Decline]

resources /ai-gateway/blob/main/site/docs/concepts/resources.md
View Suggested Changes
@@ -23,6 +23,7 @@
 A resource that defines a unified AI API for a Gateway, allowing clients to interact with multiple AI backends using a single schema.
 
 - Specifies the input API schema for client requests
+- Can optionally match traffic based on HTTP Host headers using the `hostnames` field, allowing hostname-based routing before applying other routing rules (maps directly to HTTPRoute.Spec.Hostnames in the Gateway API)
 - Contains routing rules to direct traffic to appropriate backends
 - Manages request/response transformations between different API schemas
 - Can track LLM request costs (like token usage)

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

How did I do? Any feedback?  Join Discord

@xianml xianml force-pushed the chore/gateway-hostname-cr branch 4 times, most recently from 34f8cfe to 7326736 Compare March 26, 2026 03:23
Signed-off-by: xianml <xian@bentoml.com>
@xianml xianml force-pushed the chore/gateway-hostname-cr branch from 7326736 to eb4ac51 Compare March 26, 2026 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant