Skip to content

Commit 65c4677

Browse files
Yuriy TeodorovychYuriy Teodorovych
authored andcommitted
Merge branch 'main' into yt-add-go-coverage-tests
2 parents bbe7cbc + cf3873c commit 65c4677

65 files changed

Lines changed: 2171 additions & 2409 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Reminder: openshift-ai-inference Gateway Removed from Docs
2+
3+
**Date:** 2026-03-08
4+
5+
The `openshift-ai-inference` Gateway was removed from the [MaaS setup documentation](content/install/maas-setup.md) because it was believed to be unnecessary for the current version.
6+
7+
**What was removed:**
8+
- The YAML block that created the `openshift-ai-inference` Gateway (in `openshift-ingress` namespace)
9+
- The Gateway Architecture info note that described the segregated gateway approach
10+
11+
**If you find out later that openshift-ai-inference IS needed**, restore it by:
12+
13+
1. Re-adding the Gateway YAML to `docs/content/install/maas-setup.md` in the "Create Gateway" section, before the maas-default-gateway block
14+
2. Re-adding the Gateway Architecture info note
15+
16+
The original content was:
17+
- Gateway name: `openshift-ai-inference`
18+
- Namespace: `openshift-ingress`
19+
- Infrastructure label: `serving.kserve.io/gateway: kserve-ingress-gateway`
20+
- Purpose: Standard KServe inference (vs maas-default-gateway for token auth and rate limiting)
21+
22+
**Reference:** The Gateway is still defined in `deployment/base/networking/odh/odh-gateway-api.yaml` if needed for kustomize deployments.

docs/content/advanced-administration/observability.md

Lines changed: 50 additions & 50 deletions
Large diffs are not rendered by default.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Subscription and Policy Cardinality
2+
3+
MaaSAuthPolicy and MaaSSubscription support both `groups` and `users` in their subject/owner configuration. Using `users` for many individual human users can cause cardinality issues in the rate-limiting and policy enforcement layer (Limitador, Authorino), which may impact performance and scalability.
4+
5+
**Recommendation:** Prefer `groups` for human users. Reserve the `users` field for Service Accounts and other programmatic identities where the number of distinct users remains small.
6+
7+
!!! note "See also"
8+
For configuration guidance, see [Quota and Access Configuration](../configuration-and-management/quota-and-access-configuration.md).
9+
10+
## TODO
11+
12+
- [ ] Document cardinality limits and observed impact
13+
- [ ] Provide guidance on when `users` is appropriate vs `groups`
14+
- [ ] Add monitoring and troubleshooting notes for cardinality-related issues

docs/content/api/openapi3.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../maas-api/openapi3.yaml

docs/content/architecture.md

Lines changed: 238 additions & 185 deletions
Large diffs are not rendered by default.

docs/content/configuration-and-management/group-membership-known-issues.md

Lines changed: 33 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,10 @@ When a user is removed from a group (e.g., removed from `premium-users` group) w
1010

1111
### How Group Membership Affects Access
1212

13-
1. **Token Request**: When a user requests a MaaS token, their group memberships are evaluated to determine their tier.
14-
2. **Service Account Creation**: A Service Account is created in the tier-specific namespace (e.g., `maas-default-gateway-tier-premium`).
13+
1. **Token Request**: When a user requests a MaaS token, their group memberships are evaluated to determine their subscription(s).
14+
2. **Service Account Creation**: A Service Account is created in the subscription-specific namespace (e.g., `maas-default-gateway-tier-premium` for the premium subscription).
1515
3. **Token Issuance**: The token is issued for the Service Account, not the original user.
16-
4. **Request Authorization**: Requests are authorized based on the Service Account's identity and the tier metadata cached in the AuthPolicy.
16+
4. **Request Authorization**: Requests are authorized based on the Service Account's identity and the subscription metadata cached in the AuthPolicy.
1717

1818
### Side Effects
1919

@@ -26,19 +26,19 @@ When a user is removed from a group (e.g., removed from `premium-users` group) w
2626
When a user is removed from a group, their existing MaaS tokens remain valid until expiration because:
2727

2828
- The token is a Kubernetes Service Account token, not a user token.
29-
- The Service Account continues to exist in the tier namespace.
29+
- The Service Account continues to exist in the subscription namespace.
3030
- Kubernetes TokenReview validates the Service Account, not the original user's group membership.
3131

3232
**Example Scenario**:
3333

3434
```text
3535
T+0h: User "alice" is in "premium-users" group
36-
T+0h: Alice requests a token -> Gets SA token in maas-default-gateway-tier-premium namespace
36+
T+0h: Alice requests a token -> Gets SA token in premium subscription namespace
3737
T+1h: Admin removes Alice from "premium-users" group
3838
T+1h: Alice's token is STILL VALID (expires at T+24h)
3939
T+1h: Alice can still make requests using the existing token
4040
T+24h: Token expires, Alice must request a new one
41-
T+24h: New token request -> Alice gets "free" tier (or fails if no tier matches)
41+
T+24h: New token request -> Alice gets "free" subscription (or fails if no subscription matches)
4242
```
4343

4444
**Workaround**:
@@ -53,25 +53,25 @@ curl -X DELETE "${HOST}/maas-api/v1/tokens" \
5353

5454
Note: The user must authenticate with their own token to revoke their tokens. Administrators cannot revoke tokens on behalf of other users in the current implementation.
5555

56-
#### 2. Rate Limiting Continues at Old Tier
56+
#### 2. Rate Limiting Continues at Old Subscription
5757

5858
**Impact**: Medium
5959

6060
**Description**:
6161

62-
The AuthPolicy caches the tier lookup result (default TTL: 5 minutes). After a user is removed from a group:
62+
The AuthPolicy caches the subscription lookup result (default TTL: 5 minutes). After a user is removed from a group:
6363

64-
- Requests within the cache window continue to use the old tier's rate limits.
65-
- After cache expiry, the tier is re-evaluated based on current group membership.
66-
- If the user still has a valid token but no longer belongs to any tier group, requests may fail.
64+
- Requests within the cache window continue to use the old subscription's rate limits.
65+
- After cache expiry, the subscription is re-evaluated based on current group membership.
66+
- If the user still has a valid token but no longer belongs to any subscription group, requests may fail.
6767

6868
**Example Timeline**:
6969

7070
```text
7171
T+0m: User removed from "premium-users" group
72-
T+1m: Request made -> Cached tier "premium" used -> Rate limit: 1000 tokens/min
72+
T+1m: Request made -> Cached subscription "premium" used -> Rate limit: 1000 tokens/min
7373
T+5m: Cache expires
74-
T+6m: Request made -> Tier lookup fails (no matching group) -> Request may fail with 403
74+
T+6m: Request made -> Subscription lookup fails (no matching group) -> Request may fail with 403
7575
```
7676

7777
**Workaround**:
@@ -85,10 +85,10 @@ T+6m: Request made -> Tier lookup fails (no matching group) -> Request may fai
8585

8686
**Description**:
8787

88-
When a user is removed from a group, their Service Account in the tier namespace is not automatically deleted:
88+
When a user is removed from a group, their Service Account in the subscription namespace is not automatically deleted:
8989

90-
- The Service Account remains in the tier namespace.
91-
- No new tokens can be issued for the old tier (tier lookup fails).
90+
- The Service Account remains in the subscription namespace.
91+
- No new tokens can be issued for the old subscription (subscription lookup fails).
9292
- Old tokens continue to work until expiration.
9393
- This is a cleanup artifact, not a security issue (access is controlled by RBAC and rate limiting).
9494

@@ -99,14 +99,14 @@ When a user is removed from a group, their Service Account in the tier namespace
9999
- To find the Service Account for a specific user, list and filter by the username pattern:
100100

101101
```bash
102-
# List all Service Accounts in the tier namespace
103-
kubectl get serviceaccount -n maas-default-gateway-tier-<old-tier>
102+
# List all Service Accounts in the subscription namespace
103+
kubectl get serviceaccount -n maas-default-gateway-tier-<old-subscription>
104104

105105
# Filter by username pattern (e.g., for user "alice@example.com")
106-
kubectl get serviceaccount -n maas-default-gateway-tier-<old-tier> | grep alice
106+
kubectl get serviceaccount -n maas-default-gateway-tier-<old-subscription> | grep alice
107107

108108
# Delete the identified Service Account
109-
kubectl delete serviceaccount <sa-name> -n maas-default-gateway-tier-<old-tier>
109+
kubectl delete serviceaccount <sa-name> -n maas-default-gateway-tier-<old-subscription>
110110
```
111111

112112
#### 4. User Downgrade Creates New Service Account
@@ -115,44 +115,44 @@ kubectl delete serviceaccount <sa-name> -n maas-default-gateway-tier-<old-tier>
115115

116116
**Description**:
117117

118-
When a user is moved to a lower tier (e.g., removed from `premium-users`, now only matching the `free` tier group, such as `system:authenticated` in the default configuration):
118+
When a user is moved to a lower subscription (e.g., removed from `premium-users`, now only matching the `free` subscription group, such as `system:authenticated` in the default configuration):
119119

120-
- A new Service Account is created in the new tier namespace (e.g., `maas-default-gateway-tier-free`).
121-
- The old Service Account in the premium tier namespace remains.
120+
- A new Service Account is created in the new subscription namespace (e.g., `maas-default-gateway-tier-free`).
121+
- The old Service Account in the premium subscription namespace remains.
122122
- Old premium tokens continue to work until expiration.
123-
- New token requests create tokens in the free tier namespace.
123+
- New token requests create tokens in the free subscription namespace.
124124

125125
**Example**:
126126

127127
```text
128-
Before: Alice in "premium-users" -> SA in maas-default-gateway-tier-premium
129-
After: Alice removed from "premium-users" (still matches "free" tier group)
128+
Before: Alice in "premium-users" -> SA in premium subscription namespace
129+
After: Alice removed from "premium-users" (still matches "free" subscription group)
130130
-> Old SA still exists in premium namespace
131-
-> New token request creates SA in maas-default-gateway-tier-free
131+
-> New token request creates SA in free subscription namespace
132132
-> Alice now has SAs in both namespaces
133133
```
134134

135135
**Workaround**:
136136

137137
- Revoke tokens before changing group membership to ensure clean transition.
138-
- Delete the user's Service Account manually from the old tier namespace when they change groups.
138+
- Delete the user's Service Account manually from the old subscription namespace when they change groups.
139139

140140
#### 5. Monitoring Shows Split Metrics
141141

142142
**Impact**: Low
143143

144144
**Description**:
145145

146-
If a user has tokens from multiple tiers (before and after group change):
146+
If a user has tokens from multiple subscriptions (before and after group change):
147147

148148
- Metrics are attributed to the Service Account's namespace.
149-
- Usage appears split across tier namespaces.
149+
- Usage appears split across subscription namespaces.
150150
- This is a reporting artifact and does not affect access control.
151151

152152
**Workaround**:
153153

154154
- Aggregate metrics by username label if available.
155-
- Encourage users to revoke old tokens after tier changes.
155+
- Encourage users to revoke old tokens after subscription changes.
156156

157157
### Recommended Practices
158158

@@ -162,9 +162,9 @@ If a user has tokens from multiple tiers (before and after group change):
162162

163163
3. **Use Short Token Expiration**: Shorter token lifetimes reduce the window of continued access after group removal.
164164

165-
4. **Clean Up Service Accounts**: When a user changes groups, manually delete their Service Account from the old tier namespace to prevent orphaned resources.
165+
4. **Clean Up Service Accounts**: When a user changes groups, manually delete their Service Account from the old subscription namespace to prevent orphaned resources.
166166

167167
### Related Documentation
168168

169-
- [Tier Configuration](./tier-configuration.md) - How to configure tier-to-group mappings
169+
- [Quota and Access Configuration](./quota-and-access-configuration.md) - How to configure subscription-to-group mappings
170170
- [Token Management](./token-management.md) - Understanding token lifecycle and revocation
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# MaaS Models
2+
3+
MaaS uses **MaaSModelRef** to identify model servers that live on the cluster. Each MaaSModelRef is a reference to a model server—it holds the information MaaS needs to perform authentication, authorization, and rate limiting.
4+
5+
By using a single unified object (MaaSModelRef) for all model types, MaaS can handle different kinds of model servers—each with its own backend and lifecycle—through one consistent interface. The controller uses a **provider paradigm** to distinguish between types: each model type (for example, LLMInferenceService, external APIs) has a provider that knows how to reconcile and resolve that type. Today, vLLM (via LLMInferenceService) is the supported provider; additional providers may be added in the future.
6+
7+
## The Model Reference
8+
9+
A MaaS model is a reference to a model server (for example, an LLMInferenceService or external API). The MaaS controller, running in the **control plane**, reconciles these references and gathers the information needed to route requests and enforce policies—such as the model's endpoint URL and readiness status.
10+
11+
That information is then used by MaaSSubscription and MaaSAuthPolicy to complete their logic: validating access, selecting subscriptions, and enforcing rate limits.
12+
13+
## How Model Information Is Used
14+
15+
Both **MaaSAuthPolicy** (access) and **MaaSSubscription** (quota) reference models by their **MaaSModelRef** name. They rely on the information that MaaSModelRef provides—gathered at the control plane—to:
16+
17+
- Route requests to the correct model endpoint
18+
- Validate that the user has access to the requested model
19+
- Apply the correct rate limits for that model
20+
21+
```mermaid
22+
flowchart LR
23+
subgraph Downstream ["Downstream (cluster)"]
24+
ModelServer["Model Server<br/>(e.g. LLMInferenceService)"]
25+
end
26+
27+
MaaSModelRef["MaaSModelRef"]
28+
29+
subgraph Policies ["Policies"]
30+
MaaSAuthPolicy["MaaSAuthPolicy"]
31+
MaaSSubscription["MaaSSubscription"]
32+
end
33+
34+
ModelServer -->|"1. Fetches endpoint,<br/>status, etc."| MaaSModelRef
35+
MaaSModelRef -->|"2. Feeds model info"| MaaSAuthPolicy
36+
MaaSModelRef -->|"2. Feeds model info"| MaaSSubscription
37+
38+
style MaaSModelRef fill:#1976d2,stroke:#333,stroke-width:2px,color:#fff
39+
style MaaSAuthPolicy fill:#e65100,stroke:#333,stroke-width:2px,color:#fff
40+
style MaaSSubscription fill:#e65100,stroke:#333,stroke-width:2px,color:#fff
41+
style ModelServer fill:#388e3c,stroke:#333,stroke-width:2px,color:#fff
42+
```
43+
44+
## Summary
45+
46+
- **MaaSModelRef** — Stores the reference to a model server; the controller gathers the information needed for auth and routing.
47+
- **MaaSAuthPolicy** and **MaaSSubscription** — Reference models by name and use that information to enforce access and quota.
48+
- **Control plane** — The MaaS controller reconciles model references and populates the data that policies and subscriptions depend on.
49+
50+
For configuration details and how to create and use MaaSModelRef, see [Quota and Access Configuration](quota-and-access-configuration.md) in the Administration Guide.

0 commit comments

Comments
 (0)