[Fix] Add applyguardrail for google oidc + aws guardrails#24328
[Fix] Add applyguardrail for google oidc + aws guardrails#24328shivamrawat1 wants to merge 3 commits intomainfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/base_aws_llm.py | Adds bedrock:ApplyGuardrail to the inline STS session policy in _get_google_oidc_credentials. The fix is correct and minimal — the session policy previously blocked guardrail calls even when the IAM role permitted them. No other paths in this file use inline session policies, so no further changes are needed here. |
Sequence Diagram
sequenceDiagram
participant Client
participant LiteLLM as LiteLLM (_get_google_oidc_credentials)
participant STS as AWS STS
participant Bedrock as AWS Bedrock
Client->>LiteLLM: Request with Google OIDC token
LiteLLM->>STS: AssumeRoleWithWebIdentity<br/>(RoleArn, WebIdentityToken,<br/>Policy: [InvokeModel, InvokeModelWithResponseStream, ApplyGuardrail])
Note over STS: Session policy acts as<br/>permissions boundary
STS-->>LiteLLM: Temporary credentials (AccessKey, SecretKey, SessionToken)
LiteLLM->>Bedrock: bedrock:InvokeModel (with guardrail config)
Note over Bedrock: Internally calls ApplyGuardrail
Bedrock-->>LiteLLM: Model response (guardrail applied)
LiteLLM-->>Client: Response
Last reviewed commit: "get useragent back"
Boto3/botocore and SigV4+httpx do not send User-Agent litellm/*, so the StringLike condition caused implicit deny on ApplyGuardrail and other Bedrock calls after AssumeRoleWithWebIdentity. Keep SecureTransport and Bedrock action allowlist. Made-with: Cursor
|
@greptile review with the new commit |
litellm/llms/bedrock/base_aws_llm.py
Outdated
| "WebIdentityToken": oidc_token, | ||
| "DurationSeconds": 3600, | ||
| "Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"},"StringLike":{"aws:UserAgent":"litellm/*"}}}]}', | ||
| "Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"}}}]}', |
There was a problem hiding this comment.
Silent removal of
aws:UserAgent condition
The "StringLike":{"aws:UserAgent":"litellm/*"} condition was silently dropped from the session policy as part of this change. The old policy included:
"Condition": {
"Bool": { "aws:SecureTransport": "true" },
"StringLike": { "aws:UserAgent": "litellm/*" }
}The new policy only retains:
"Condition": {
"Bool": { "aws:SecureTransport": "true" }
}This is an undocumented behavioral change. While aws:UserAgent is indeed not recommended by AWS as a reliable security control (it can be spoofed), the original policy explicitly scoped calls to a litellm/* user-agent as a defense-in-depth measure.
If the removal is intentional (e.g., to avoid breakage when the guardrail call uses a different User-Agent), it should be documented in the PR description. If it is accidental, the condition should be restored:
| "Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"}}}]}', | |
| "Policy": '{"Version":"2012-10-17","Statement":[{"Sid":"BedrockLiteLLM","Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream","bedrock:ApplyGuardrail"],"Resource":"*","Condition":{"Bool":{"aws:SecureTransport":"true"},"StringLike":{"aws:UserAgent":"litellm/*"}}}]}', |
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes