Skip to content

feat: remove identity headers and simplify subscription info#645

Merged
jrhyness merged 7 commits intoopendatahub-io:mainfrom
jrhyness:jr_55482_55480
Mar 31, 2026
Merged

feat: remove identity headers and simplify subscription info#645
jrhyness merged 7 commits intoopendatahub-io:mainfrom
jrhyness:jr_55482_55480

Conversation

@jrhyness
Copy link
Copy Markdown
Contributor

@jrhyness jrhyness commented Mar 30, 2026

Description

Remove identity headers from model inference requests and simplify subscription-info handling in controller-generated AuthPolicies (defense-in-depth security improvements).

Changes:

  • Remove X-MaaS-Username, X-MaaS-Group, and X-MaaS-Key-Id headers from requests forwarded to model workloads
  • Simplify AuthPolicy by passing complete subscription-info object instead of extracting individual fields
  • Fix CEL syntax error in subscription_info has() check that was blocking AuthPolicy enforcement
  • Identity data remains available to TokenRateLimitPolicy and telemetry via filters.identity mechanism

Security Motivation:

Model workloads (vLLM, Llama.cpp, external APIs) do not require identity headers. Removing them prevents accidental disclosure in:

  • Model runtime logs
  • Upstream debug dumps
  • Misconfigured proxies or sidecars
  • External provider logs (for ExternalModel backends)

Scope:

Only affects controller-generated AuthPolicies for model inference routes. The static maas-api AuthPolicy is unchanged (maas-api's ExtractUserInfo middleware still requires headers).

Subscription Info Changes:

Before (multiple extracted fields):

filters:                                                                                                                                                                              
  identity:                                                                                                                                                                           
    - organizationId: auth.metadata["subscription-info"].organizationId
    - costCenter: auth.metadata["subscription-info"].costCenter                                                                                                                       
    - subscription_labels: auth.metadata["subscription-info"].labels                                                                                                                  
                                                                                                                                                                                      
After (single object):                                                                                                                                                                
filters:                                                                                                                                                                              
  identity:                                                                                                                                                                           
    - subscription_info: auth.metadata["subscription-info"]
                                                                                                                                                                                      
Consumers update expressions from auth.identity.organizationId to auth.identity.subscription_info.organizationId.                                                                     
                                                                                                                                                                                      
How Has This Been Tested?                                                                                                                                                             
                                                                                                                                                                                      
Unit Tests:                                                                                                                                                                           
                                                          
1. TestMaaSAuthPolicyReconciler_IdentityHeadersNotForwarded                                                                                                                           
  - Verifies headers (X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id) are NOT in response.success.headers
  - Confirms identity data IS available via filters.identity for TRLP/telemetry                                                                                                       
  - Ensures defense-in-depth without breaking rate limiting or metrics                                                                                                                
2. TestMaaSAuthPolicyReconciler_SubscriptionInfoObject                                                                                                                                
  - Verifies subscription_info field returns complete object (not extracted fields)                                                                                                   
  - Confirms organizationId, costCenter, and labels are accessible via nested paths                                                                                                   
  - Tests has() check uses correct CEL syntax (subscription-info.name)                                                                                                                
                                                                                                                                                                                      
Integration Testing:                                                                                                                                                                  
                                                                                                                                                                                      
- ✅ All existing controller tests pass (no regressions)                                                                                                                              
- ✅ CEL syntax fix unblocks AuthPolicy enforcement (Enforced=True)
- ✅ TelemetryPolicy updated to use nested subscription_info fields                                                                                                                   
                                                                                                                                                                                      
Manual Testing:                                                                                                                                                                       
                                                                                                                                                                                      
Smoke tests were blocked by CEL syntax error; fixed in this PR. CI pipeline will validate full e2e flows.                                                                             
                                                          
Merge criteria:                                                                                                                                                                       
                                                          
- The commits are squashed in a cohesive manner and have meaningful messages.                                                                                                         
- Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
- The developer has manually tested the changes and verified that the changes work   


For: 
https://redhat.atlassian.net/browse/RHOAIENG-55482
https://redhat.atlassian.net/browse/RHOAIENG-55480


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **Bug Fixes**
* Identity headers are no longer forwarded to upstream model workloads; gateway-only consumption enforced.
* Telemetry label sources updated to use subscription_info fields for metrics.

* **Documentation**
* Clarified identity propagation and defense-in-depth: what is exposed to gateway telemetry and rate limiting, which identity headers are not forwarded, and where subscription data is available.

* **Tests**
* Added reconciliation test verifying no identity headers upstream and required subscription telemetry.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

jrhyness and others added 4 commits March 30, 2026 13:07
Remove X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id, and X-MaaS-Subscription
headers from requests forwarded to upstream model workloads (defense-in-depth).

All identity information remains available to TokenRateLimitPolicy and gateway
telemetry through AuthPolicy's filters.identity mechanism, but is not exposed
to model runtime pods to prevent accidental disclosure in logs or dumps.

Changes:
- Remove response.success.headers from controller-generated AuthPolicies
- Add test verifying headers not forwarded and identity data available
- Update docs to explain security model and scope (model routes only)

Out of scope: maas-api static AuthPolicy unchanged (still uses headers for
ExtractUserInfo middleware on maas-api-route).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Pass the complete subscription-info object instead of extracting individual
fields, simplifying the AuthPolicy and enabling consumers to access any field
without controller changes.

Changes:
- Add subscription_info field with full auth.metadata["subscription-info"] object
- Remove redundant top-level fields:
  - organizationId (now: subscription_info.organizationId)
  - costCenter (now: subscription_info.costCenter)
  - subscription_labels (now: subscription_info.labels)
- Update TelemetryPolicy to access nested fields
- Add test coverage verifying subscription_info returns full object

Benefits:
- Cleaner policy: single object vs multiple extracted fields
- Extensibility: new subscription fields automatically available
- Consistency: field name matches source (subscription-info → subscription_info)

Consumers should update expressions:
  Before: auth.identity.organizationId
  After:  auth.identity.subscription_info.organizationId

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fix CEL syntax error in AuthPolicy subscription_info field that was causing
AuthConfig validation failures and blocking model access.

The has() macro in Authorino/Kuadrant CEL does not support map subscript
notation has(auth.metadata["subscription-info"]). Changed to check for a
specific field within the object: has(auth.metadata["subscription-info"].name).

Error message before fix:
  ERROR: <input>:1:18: invalid argument to has() macro
   | has(auth.metadata["subscription-info"]) ? ...
   | .................^

This was causing AuthPolicies to remain in Enforced=False state, preventing
smoke tests from passing.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Keep X-MaaS-Subscription header injection for Istio Telemetry's per-subscription
latency tracking. Other identity headers (Username, Group, Key-Id) remain removed
for defense-in-depth.

Istio Telemetry runs in Envoy and cannot access auth.identity context - it can
only read request headers. The X-MaaS-Subscription header is server-controlled
(injected by Authorino from validated subscription), not client-provided.

Updates test to verify X-MaaS-Subscription is present while other identity
headers remain absent.
@openshift-ci openshift-ci bot requested review from SB159 and chaitanya1731 March 30, 2026 17:22
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 30, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 40efc5e0-c2ba-407b-8f97-2071940a19fa

📥 Commits

Reviewing files that changed from the base of the PR and between 1a42e9b and 9d5919d.

📒 Files selected for processing (2)
  • docs/content/configuration-and-management/maas-controller-overview.md
  • maas-controller/pkg/controller/maas/maasauthpolicy_controller.go
✅ Files skipped from review due to trivial changes (1)
  • docs/content/configuration-and-management/maas-controller-overview.md

📝 Walkthrough

Walkthrough

Removed forwarding of identity headers (X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id) from generated Kuadrant AuthPolicy targeting upstream model workloads; retained X-MaaS-Subscription response header for Istio telemetry (value populated from API key validation metadata, empty-string fallback for K8s tokens). TelemetryPolicy metric label mappings updated to use auth.identity.subscription_info.organizationId and auth.identity.subscription_info.costCenter. AuthPolicy filters.identity shape now exposes a subscription_info object (whole auth.metadata["subscription-info"] when present) instead of extracting organizationId, costCenter, or subscription_labels. Documentation updated to state identity is consumed at the gateway (Authorino auth.identity / auth.metadata) for TokenRateLimitPolicy, telemetry, and OPA; controller-generated AuthPolicies do not inject identity headers to model workloads (maas-api routes remain unchanged). Added a unit test asserting no identity headers are forwarded upstream and that subscription_info is exposed for telemetry and metrics.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Security & Actionable Issues

  • Removing identity headers reduces information exposure (CWE-200). Action: audit upstream consumers for reliance on X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id; plan migration or add explicit allowlist exceptions where strictly required.
  • X-MaaS-Subscription may be emitted as an empty string for K8s-token flows — risk of treating absent subscription as valid (CWE-287: Improper Authentication). Action: enforce upstream validation that treats empty/absent subscription as failure; fail closed.
  • Telemetry label path change to auth.identity.subscription_info.* can drop or corrupt metrics if subscription_info is missing/malformed (affects billing/aggregation). Action: add defensive mapping defaults in TelemetryPolicy or ensure Authorino always emits a normalized subscription_info object; add unit/integration tests for absent/null/malformed subscription-info.
  • Reliance on Authorino contexts (auth.identity / auth.metadata) without sanitization risks malformed inputs affecting policy evaluation (CWE-20: Improper Input Validation). Action: add CEL guards and validation asserting expected shapes/types before using fields in filters.identity and metrics.
  • Tests: new unit test verifies header removal and presence of subscription_info but does not cover edge cases (missing/malformed subscription-info, metrics label fallbacks, empty subscription header). Action: add tests for absent/null/malformed subscription-info, empty X-MaaS-Subscription, and ensure metric-label fallback behavior is validated.
  • Recommend adding logging/metrics around identity-to-telemetry mappings to detect missing labels or unexpected defaults early.

CVE references: none directly applicable to these configuration/documentation/test changes.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the two main changes: removing identity headers (X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id) and simplifying subscription info structure (flattening to subscription_info object).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/content/configuration-and-management/maas-controller-overview.md`:
- Line 238: Docs claim X-MaaS-Subscription is not injected but code in
maasauthpolicy_controller.go (around the injection logic at lines handling Istio
telemetry, e.g., the block at maasauthpolicy_controller.go:462-468) and the unit
test maasauthpolicy_controller_test.go (assertions at ~1120-1123) show
X-MaaS-Subscription is injected; update the documentation text to reflect that
X-MaaS-Subscription is injected for Istio telemetry/per-subscription latency
tracking, and add the proposed clarifying sentence after the bullet list to
state this exception and its purpose.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: d1e42bf8-5820-4e26-9ef5-9bc6eb42f154

📥 Commits

Reviewing files that changed from the base of the PR and between 4f06bcf and 602fc04.

📒 Files selected for processing (5)
  • deployment/base/observability/gateway-telemetry-policy.yaml
  • docs/content/architecture.md
  • docs/content/configuration-and-management/maas-controller-overview.md
  • maas-controller/pkg/controller/maas/maasauthpolicy_controller.go
  • maas-controller/pkg/controller/maas/maasauthpolicy_controller_test.go

jrhyness and others added 2 commits March 30, 2026 13:34
Update documentation to reflect that X-MaaS-Subscription header IS injected
(exception to the identity header removal), while other identity headers
(Username, Group, Key-Id) remain removed.

Explains why Istio requires the header (cannot access auth.identity context)
and clarifies the value is server-controlled, not client-provided.
Use apiKeyValidation.subscription (bound) instead of subscription-info.name
(selected) for the X-MaaS-Subscription header. For K8s tokens, the expression
returns empty string so no header is injected. This aligns with actual usage:
API keys for inference (needs header for Istio Telemetry), K8s tokens for
/v1/models catalog browsing (no subscription tracking needed).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@jrhyness jrhyness added 3.4 ready-for-review All changes are complete and CI checks have passed. labels Mar 30, 2026
Merges upstream changes from main while preserving the simplified
AuthPolicy headers approach from this PR.

Conflict resolution:
- Kept Authorization header stripping from PR opendatahub-io#643
- Kept X-MaaS-Subscription header (for Istio Telemetry)
- Removed X-MaaS-Username, X-MaaS-Group, X-MaaS-Key-Id headers
  (as intended by this PR's simplification)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@jrhyness jrhyness enabled auto-merge (squash) March 31, 2026 15:39
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 31, 2026

@jrhyness: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-test-maas 9d5919d link false /test e2e-test-maas

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link
Copy Markdown
Contributor

@jland-redhat jland-redhat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jrhyness jrhyness merged commit 1221310 into opendatahub-io:main Mar 31, 2026
14 of 16 checks passed
@openshift-ci openshift-ci bot added the lgtm label Mar 31, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 31, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jland-redhat, jrhyness

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jrhyness jrhyness deleted the jr_55482_55480 branch March 31, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.4 approved lgtm ready-for-review All changes are complete and CI checks have passed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants