Skip to content

Commit b8952dd

Browse files
Yuriy TeodorovychYuriy Teodorovych
authored andcommitted
Merge branch 'main' into yt-standardize-crds
2 parents 0571c64 + 8b3c6c1 commit b8952dd

File tree

9 files changed

+28
-43
lines changed

9 files changed

+28
-43
lines changed

docs/content/configuration-and-management/maas-controller-overview.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ flowchart TB
5757

5858
**Summary:** You declare intent with MaaS CRs; the controller turns that into Gateway/Kuadrant resources that attach to the same HTTPRoute and backend (e.g. KServe LLMInferenceService).
5959

60-
The **MaaS API** GET /v1/models endpoint uses MaaSModelRef CRs as its primary source: it lists them in the API namespace, then **validates access** by probing each model’s `/v1/models` endpoint with the client’s **Authorization header** (passed through as-is). Only models that return 2xx or 405 are included. So the catalogue returned to the client is the set of MaaSModelRef objects the controller reconciles, filtered to those the client can actually access. No token exchange is performed; the header is forwarded as-is. (Once minting is in place, this may be revisited.)
60+
The **MaaS API** GET /v1/models endpoint uses MaaSModelRef CRs as its primary source: it reads them cluster-wide (all namespaces), then **validates access** by probing each model’s `/v1/models` endpoint with the client’s **Authorization header** (passed through as-is). Only models that return 2xx or 405 are included. So the catalogue returned to the client is the set of MaaSModelRef objects the controller reconciles, filtered to those the client can actually access. No token exchange is performed; the header is forwarded as-is.
6161

6262
---
6363

@@ -213,7 +213,7 @@ flowchart LR
213213
Deploy --> Examples
214214
```
215215

216-
- **Namespace**: Controller and default MaaS CRs live in **opendatahub** (configurable).
216+
- **Namespaces**: MaaS API and controller default to **opendatahub** (configurable). MaaSAuthPolicy and MaaSSubscription default to **models-as-a-service** (configurable). MaaSModelRef must live in the **same namespace** as the model it references (e.g. **llm**).
217217
- **Install**: `./scripts/deploy.sh` installs the full stack including the controller. Optionally run `./scripts/install-examples.sh` for sample MaaSModelRef, MaaSAuthPolicy, and MaaSSubscription.
218218

219219
---
@@ -233,7 +233,7 @@ The Kuadrant AuthPolicy validates API keys via the MaaS API and validates user t
233233
| Topic | Summary |
234234
|-------|---------|
235235
| **What** | MaaS Controller = control plane that reconciles MaaSModelRef, MaaSAuthPolicy, and MaaSSubscription into Gateway API and Kuadrant resources. |
236-
| **Where** | Single controller in `maas-controller`; CRs and generated resources can live in opendatahub or other namespaces. |
236+
| **Where** | Single controller in `opendatahub`; MaaSAuthPolicy / MaaSSubscription default to `models-as-a-service`; MaaSModelRef and generated Kuadrant policies target their model’s namespace. |
237237
| **How** | Three reconcilers watch MaaS CRs (and related resources); each creates/updates HTTPRoutes, AuthPolicies, or TokenRateLimitPolicies. |
238238
| **Identity bridge** | AuthPolicy exposes all user groups as a comma-separated `groups_str`; TokenRateLimitPolicy uses `groups_str.split(",").exists(...)` for subscription matching (the “string trick”). |
239239
| **Deploy** | Run `./scripts/deploy.sh`; optionally install examples. |

docs/content/configuration-and-management/maas-model-kinds.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ apiVersion: maas.opendatahub.io/v1alpha1
3131
kind: MaaSModelRef
3232
metadata:
3333
name: my-model
34-
namespace: opendatahub
34+
namespace: llm
3535
spec:
3636
modelRef:
3737
kind: LLMInferenceService
@@ -63,7 +63,7 @@ The controller:
6363

6464
## API Behavior
6565

66-
- The API reads MaaSModelRefs from the informer cache, maps each to an API model (`id`, `url`, `ready`, `kind`, etc.)
66+
- The API reads MaaSModelRefs cluster-wide, maps each to an API model (`id`, `url`, `ready`, `kind`, etc.)
6767
- **Access validation**: Probes each model's `/v1/models` endpoint with the request's Authorization header. Only models that return 2xx or 405 are included.
6868
- **Kind on the wire**: Each model in the GET /v1/models response carries a `kind` field from `spec.modelRef.kind`
6969

@@ -92,6 +92,6 @@ To support a new backend type (a new **kind** in `spec.modelRef`):
9292
## Summary
9393

9494
- **modelRef** is the backend reference (kind, name, optional namespace), analogous to [Gateway API BackendRef](https://gateway-api.sigs.k8s.io/reference/spec/#backendref).
95-
- **Listing:** Always from MaaSModelRef cache; no kind-specific listing logic.
95+
- **Listing:** Always from MaaSModelRef resources cluster-wide; no kind-specific listing logic.
9696
- **Access validation:** Same probe (GET endpoint with the request's Authorization header as-is) for all kinds unless kind-specific probes are added later.
9797
- **New kinds:** Implement in the controller (resolve referent, set status.endpoint and status.phase); extend the API only if the new kind cannot use the same probe path or needs different enrichment.

docs/content/configuration-and-management/model-listing-flow.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
This document describes how the **GET /v1/models** endpoint discovers and returns the list of available models.
44

5-
The list is **based on MaaSModelRef** custom resources: the API returns models that are registered as MaaSModelRefs in its configured namespace.
5+
The list is **based on MaaSModelRef** custom resources: the API considers MaaSModelRef objects cluster-wide (all namespaces), then filters by access.
66

77
## Overview
88

@@ -17,9 +17,9 @@ Each entry includes an `id`, **`url`** (the model’s endpoint), a `ready` flag,
1717

1818
## MaaSModelRef flow
1919

20-
When the [MaaS controller](https://github.com/opendatahub-io/models-as-a-service/tree/main/maas-controller) is installed and the API is configured with a MaaSModelRef lister and namespace, the flow is:
20+
When the [MaaS controller](https://github.com/opendatahub-io/models-as-a-service/tree/main/maas-controller) is installed and the API is configured with a MaaSModelRef lister, the flow is:
2121

22-
1. The MaaS API lists all **MaaSModelRef** custom resources in its configured namespace (e.g. `opendatahub`). It reads them from an **in-memory cache** in the maas-api component (maintained by a Kubernetes informer), so it does not call the Kubernetes API on every request.
22+
1. The MaaS API discovers **MaaSModelRef** custom resources **cluster-wide** (all namespaces) without calling the Kubernetes API on every request.
2323

2424
2. For each MaaSModelRef, it reads **id** (`metadata.name`), **url** (`status.endpoint`), **ready** (`status.phase == "Ready"`), and related metadata. The controller has populated `status.endpoint` and `status.phase` from the underlying LLMInferenceService (for llmisvc) or HTTPRoute/Gateway.
2525

@@ -55,7 +55,7 @@ sequenceDiagram
5555

5656
- **Consistent with gateway**: The same model names and routes are used for inference; the list matches what the gateway will accept for that client.
5757

58-
If the API is not configured with a MaaSModelRef lister and namespace, or if listing fails (e.g. CRD not installed, no RBAC, or server error), the API returns an empty list or an error.
58+
If the API is not configured with a MaaSModelRef lister, or if listing fails (e.g. CRD not installed, no RBAC, or server error), the API returns an empty list or an error.
5959

6060
## Subscription Filtering and Aggregation
6161

@@ -174,7 +174,7 @@ To have models appear via the **MaaSModelRef** flow:
174174
kind: MaaSModelRef
175175
metadata:
176176
name: my-model-name # This becomes the model "id" in GET /v1/models
177-
namespace: opendatahub
177+
namespace: llm # Same namespace as the LLMInferenceService
178178
annotations:
179179
openshift.io/display-name: "My Model" # optional: human-readable name
180180
openshift.io/description: "A general-purpose LLM" # optional: description
@@ -184,9 +184,8 @@ To have models appear via the **MaaSModelRef** flow:
184184
modelRef:
185185
kind: LLMInferenceService
186186
name: my-llm-isvc-name
187-
namespace: llm
188187

189-
4. The controller reconciles the MaaSModelRef and sets `status.endpoint` and `status.phase`. The MaaS API (in the same namespace) will then include this model in GET /v1/models when it lists MaaSModelRef CRs.
188+
4. The controller reconciles the MaaSModelRef and sets `status.endpoint` and `status.phase`. The MaaS API will then include this model in GET /v1/models when it lists MaaSModelRef CRs.
190189

191190
You can use the [maas-system samples](https://github.com/opendatahub-io/models-as-a-service/tree/main/docs/samples/maas-system) as a template; the install script deploys LLMInferenceService + MaaSModelRef + MaaSAuthPolicy + MaaSSubscription together so dependencies resolve correctly.
192191

docs/content/install/model-setup.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,11 +91,11 @@ kubectl get pods -n llm
9191
**Validate MaaSModelRef status** — The MaaS controller populates `status.endpoint` and `status.phase` on each MaaSModelRef from the LLMInferenceService. The MaaSModelRef `status.endpoint` should match the URL exposed by the LLMInferenceService (via the gateway). Verify:
9292

9393
```bash
94-
# Check MaaSModelRef status (use opendatahub for ODH, redhat-ods-applications for RHOAI)
95-
kubectl get maasmodelref -n opendatahub -o wide
94+
# Check MaaSModelRef status (same namespace as the LLMInferenceService, e.g. llm)
95+
kubectl get maasmodelref -n llm -o wide
9696

9797
# Verify status.endpoint is populated and phase is Ready
98-
kubectl get maasmodelref -n opendatahub -o jsonpath='{range .items[*]}{.metadata.name}: phase={.status.phase} endpoint={.status.endpoint}{"\n"}{end}'
98+
kubectl get maasmodelref -n llm -o jsonpath='{range .items[*]}{.metadata.name}: phase={.status.phase} endpoint={.status.endpoint}{"\n"}{end}'
9999

100100
# Compare with LLMInferenceService — status.endpoint should match the URL from LLMIS status.addresses or status.url
101101
kubectl get llminferenceservice -n llm -o yaml | grep "url:"

docs/content/reference/crds/maas-model-ref.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# MaaSModelRef
22

3-
Identifies an AI/ML model on the cluster. The MaaS API lists models from MaaSModelRef resources (using `status.endpoint` and `status.phase`).
3+
Identifies an AI/ML model on the cluster. Create MaaSModelRef in the **same namespace** as the backend (`LLMInferenceService`, `ExternalModel`, etc.). The MaaS API lists models from MaaSModelRef resources cluster-wide (using `status.endpoint` and `status.phase`).
44

55
## MaaSModelRefSpec
66

docs/samples/maas-system/README.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,9 @@ kustomize build docs/samples/maas-system/ | kubectl apply -f -
2525
# Or deploy a specific sample
2626
kustomize build docs/samples/maas-system/facebook-opt-125m-cpu/ | kubectl apply -f -
2727
kustomize build docs/samples/maas-system/qwen3/ | kubectl apply -f -
28-
```
2928

3029
# Verify
31-
kubectl get maasmodelref -n opendatahub
30+
kubectl get maasmodelref -n llm
3231
kubectl get maasauthpolicy,maassubscription -n models-as-a-service
3332
kubectl get llminferenceservice -n llm
3433
```
@@ -44,7 +43,7 @@ kubectl create namespace llm --dry-run=client -o yaml | kubectl apply -f -
4443
kustomize build docs/samples/maas-system | sed "s/namespace: models-as-a-service/namespace: my-namespace/g" | kubectl apply -f -
4544

4645
# Verify
47-
kubectl get maasmodelref -n opendatahub
46+
kubectl get maasmodelref -n llm
4847
kubectl get maasauthpolicy,maassubscription -n my-namespace
4948
kubectl get llminferenceservice -n llm
5049
```

maas-controller/README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,8 @@ Models with no MaaSAuthPolicy or MaaSSubscription are denied at the gateway leve
3535
### CRDs and what they generate
3636

3737
As MaaS API and controller are conventionally deployed in the operator namespace (e.g., `opendatahub`), MaaS CRs need to be separated so that they can be managed with lower cluster privileges. Therefore,
38-
- **MaasModelRef** is located in the same namespace as the **HTTPRoute** and **LLMInderenceService** it refers to; and
39-
- **MaaSAuthPolicy** and **MaaSSubscription** are located in a dedicated subscription namespace (default: `models-as-a-service`). Set `--maas-subscription-namespace` or the `MAAS_SUBSCRIPTION_NAMESPACE` env var in `maas-controller` deployment to use another namespace. MaaS controller will only watch and reconcile those CRs this configured namespace.
38+
- **MaaSModelRef** is located in the same namespace as the **HTTPRoute** and **LLMInferenceService** it refers to; and
39+
- **MaaSAuthPolicy** and **MaaSSubscription** are located in a dedicated subscription namespace (default: `models-as-a-service`). Set `--maas-subscription-namespace` or the `MAAS_SUBSCRIPTION_NAMESPACE` env var in `maas-controller` deployment to use another namespace. MaaS controller will only watch and reconcile those CRs in this configured namespace.
4040

4141
| You create | Controller generates | Per | Targets |
4242
| ---------- | -------------------- | --- | ------- |
@@ -66,12 +66,12 @@ spec:
6666
kind: LLMInferenceService
6767
name: my-llmisvc
6868
---
69-
# MaaSAuthPolicy in opendatahub namespace references model in llm namespace
69+
# MaaSAuthPolicy in models-as-a-service namespace references model in llm namespace
7070
apiVersion: maas.opendatahub.io/v1alpha1
7171
kind: MaaSAuthPolicy
7272
metadata:
7373
name: my-policy
74-
namespace: opendatahub
74+
namespace: models-as-a-service
7575
spec:
7676
modelRefs:
7777
- name: my-model
@@ -81,7 +81,7 @@ spec:
8181
- name: my-group
8282
```
8383
84-
The controller creates a Kuadrant **AuthPolicy** in the `llm` namespace (where the model and HTTPRoute exist), not in `opendatahub` (where the MaaSAuthPolicy lives).
84+
The controller creates a Kuadrant **AuthPolicy** in the `llm` namespace (where the model and HTTPRoute exist), not in `models-as-a-service` (where the MaaSAuthPolicy lives).
8585

8686
**Same model name, different namespaces:**
8787

@@ -99,7 +99,7 @@ spec:
9999

100100
This creates two separate AuthPolicies: one in `team-a`, one in `team-b`.
101101

102-
**Model list API:** When the MaaS controller is installed, the MaaS API **GET /v1/models** endpoint lists models by reading **MaaSModelRef** CRs (in the API's namespace). Each MaaSModelRef's `metadata.name` becomes the model `id`, and `status.endpoint` / `status.phase` supply the URL and readiness. So the set of MaaSModelRef objects is the source of truth for "which models are available" in MaaS. See [docs/content/configuration-and-management/model-listing-flow.md](../docs/content/configuration-and-management/model-listing-flow.md) in the repo for the full flow.
102+
**Model list API:** When the MaaS controller is installed, the MaaS API **GET /v1/models** endpoint lists models by reading **MaaSModelRef** CRs cluster-wide (all namespaces). Each MaaSModelRef's `metadata.name` becomes the model `id`, and `status.endpoint` / `status.phase` supply the URL and readiness. So the set of MaaSModelRef objects is the source of truth for "which models are available" in MaaS. See [docs/content/configuration-and-management/model-listing-flow.md](../docs/content/configuration-and-management/model-listing-flow.md) in the repo for the full flow.
103103

104104
### Model kinds and the provider pattern
105105

@@ -270,13 +270,13 @@ This creates:
270270
### Regular tier
271271

272272
- `LLMInferenceService/facebook-opt-125m-simulated` in `llm` namespace
273-
- `MaaSModelRef/facebook-opt-125m-simulated` in `opendatahub`
273+
- `MaaSModelRef/facebook-opt-125m-simulated` in `llm`
274274
- `MaaSAuthPolicy/simulator-access` (group: `free-user`) and `MaaSSubscription/simulator-subscription` (100 tokens/min) in `models-as-a-service`
275275

276276
### Premium tier
277277

278278
- `LLMInferenceService/premium-simulated-simulated-premium` in `llm` namespace
279-
- `MaaSModelRef/premium-simulated-simulated-premium` in `opendatahub`
279+
- `MaaSModelRef/premium-simulated-simulated-premium` in `llm`
280280
- `MaaSAuthPolicy/premium-simulator-access` (group: `premium-user`) and `MaaSSubscription/premium-simulator-subscription` (1000 tokens/min) in `models-as-a-service`
281281

282282
Replace `free-user` and `premium-user` in the example CRs with groups from your identity provider.
@@ -285,7 +285,7 @@ Then verify:
285285

286286
```bash
287287
# Check CRs
288-
kubectl get maasmodelref -n opendatahub
288+
kubectl get maasmodelref -n llm
289289
kubectl get maasauthpolicy,maassubscription -n models-as-a-service
290290
291291
# Check generated Kuadrant policies

maas-controller/docs/old-vs-new-flow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ Model Endpoint
9898

9999
| Resource | Namespace | Created by | Purpose |
100100
|----------|-----------|------------|---------|
101-
| MaaSModelRef | opendatahub | User/admin | Registers a model with MaaS |
101+
| MaaSModelRef | Same as model (e.g. `llm`) | User/admin | Registers a model with MaaS |
102102
| MaaSAuthPolicy | models-as-a-service | User/admin | Defines who (groups) can access which models |
103103
| MaaSSubscription | models-as-a-service | User/admin | Defines per-model token rate limits for owner groups |
104104
| AuthPolicy (generated) | llm | maas-controller | Per-model auth, one per (MaaSAuthPolicy, model) pair |

maas-controller/examples/maas-model.yaml

Lines changed: 0 additions & 13 deletions
This file was deleted.

0 commit comments

Comments
 (0)