Skip to content

[Fix] Add applyguardrail for google oidc + aws guardrails#24328

Open
shivamrawat1 wants to merge 3 commits intomainfrom
litellm_applyguardrail_oidc
Open

[Fix] Add applyguardrail for google oidc + aws guardrails#24328
shivamrawat1 wants to merge 3 commits intomainfrom
litellm_applyguardrail_oidc

Conversation

@shivamrawat1
Copy link
Collaborator

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

@vercel
Copy link

vercel bot commented Mar 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Mar 21, 2026 11:04pm

Request Review

@codspeed-hq
Copy link
Contributor

codspeed-hq bot commented Mar 21, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_applyguardrail_oidc (7a0d86b) with main (e3d4c29)

Open in CodSpeed

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 21, 2026

Greptile Summary

This PR adds bedrock:ApplyGuardrail to the inline STS session policy used in _get_google_oidc_credentials. The inline session policy acts as a permissions boundary when assuming a role via AssumeRoleWithWebIdentity — it restricts what the resulting session credentials are allowed to do. By adding bedrock:ApplyGuardrail to the allowed actions, users relying on Google OIDC authentication can now make guardrail-related Bedrock API calls (provided the IAM role itself also permits them).

  • The change is a single-line fix in litellm/llms/bedrock/base_aws_llm.py, targeting only the _get_google_oidc_credentials method
  • Other credential paths (_auth_with_aws_role, _handle_irsa_cross_account, _handle_irsa_same_account) do not use inline session policies and thus were not affected by this restriction
  • The aws:UserAgent condition (StringLike: litellm/*) is unchanged from the original policy
  • The PR description leaves the issue reference as #000 and the testing checklist item is unchecked — there is no evidence provided that the issue is resolved or verifiable

Confidence Score: 3/5

  • The code change itself is correct and minimal, but the absence of any linked issue or tests makes the fix hard to verify.
  • The one-line change is logically correct — adding bedrock:ApplyGuardrail to the inline STS session policy is the right fix to unblock guardrail calls for Google OIDC credential flows. However, the PR has no linked issue, no description of the problem being solved, and the pre-submission testing checklist is left unchecked, making it difficult to confirm the fix actually resolves a real reported regression.
  • No files require special attention given the narrowness of the change, but litellm/llms/bedrock/base_aws_llm.py line 703 (the hardcoded policy string) would benefit from a unit test.

Important Files Changed

Filename Overview
litellm/llms/bedrock/base_aws_llm.py Adds bedrock:ApplyGuardrail to the inline STS session policy in _get_google_oidc_credentials. The fix is correct and minimal — the session policy previously blocked guardrail calls even when the IAM role permitted them. No other paths in this file use inline session policies, so no further changes are needed here.

Sequence Diagram

sequenceDiagram
    participant Client
    participant LiteLLM as LiteLLM (_get_google_oidc_credentials)
    participant STS as AWS STS
    participant Bedrock as AWS Bedrock

    Client->>LiteLLM: Request with Google OIDC token
    LiteLLM->>STS: AssumeRoleWithWebIdentity<br/>(RoleArn, WebIdentityToken,<br/>Policy: [InvokeModel, InvokeModelWithResponseStream, ApplyGuardrail])
    Note over STS: Session policy acts as<br/>permissions boundary
    STS-->>LiteLLM: Temporary credentials (AccessKey, SecretKey, SessionToken)
    LiteLLM->>Bedrock: bedrock:InvokeModel (with guardrail config)
    Note over Bedrock: Internally calls ApplyGuardrail
    Bedrock-->>LiteLLM: Model response (guardrail applied)
    LiteLLM-->>Client: Response
Loading

Last reviewed commit: "get useragent back"

Boto3/botocore and SigV4+httpx do not send User-Agent litellm/*, so the
StringLike condition caused implicit deny on ApplyGuardrail and other
Bedrock calls after AssumeRoleWithWebIdentity. Keep SecureTransport and
Bedrock action allowlist.

Made-with: Cursor
@shivamrawat1
Copy link
Collaborator Author

@greptile review with the new commit

"WebIdentityToken": oidc_token,
"DurationSeconds": 3600,
"Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"},"StringLike":{"aws:UserAgent":"litellm/*"}}}]}',
"Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"}}}]}',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Silent removal of aws:UserAgent condition

The "StringLike":{"aws:UserAgent":"litellm/*"} condition was silently dropped from the session policy as part of this change. The old policy included:

"Condition": {
  "Bool":   { "aws:SecureTransport": "true" },
  "StringLike": { "aws:UserAgent": "litellm/*" }
}

The new policy only retains:

"Condition": {
  "Bool": { "aws:SecureTransport": "true" }
}

This is an undocumented behavioral change. While aws:UserAgent is indeed not recommended by AWS as a reliable security control (it can be spoofed), the original policy explicitly scoped calls to a litellm/* user-agent as a defense-in-depth measure.

If the removal is intentional (e.g., to avoid breakage when the guardrail call uses a different User-Agent), it should be documented in the PR description. If it is accidental, the condition should be restored:

Suggested change
"Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"}}}]}',
"Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"},"StringLike":{"aws:UserAgent":"litellm/*"}}}]}',

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant