Skip to content

ExpiredTokenException not triggering retrys #2962

Closed as not planned
Closed as not planned
@torchsec-roni

Description

@torchsec-roni

Acknowledgements

Describe the bug

Our production code uses STS authenticate with web identity (i.e. auth-via-JWT).
We are encountering a race condition where the Client performs a check whether the JWT is expired at t0 and then forwards the request to STS at t1. At t0 the JWT is not expired, at t1 it is. In this case, a retry would trigger the SDK to go and fetch a new JWT, but because ExpiredTokenException is not a retryable error by default, the SDK just errors out.

I believe this was once a retryable error as per aws/aws-sdk-go#212 but it looks like it isn't anymore: https://github.com/aws/aws-sdk-go-v2/blob/main/aws/retry/standard.go#L79

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

The SDK retries on ExpiredTokenException

Current Behavior

The SDK does not retry on ExpiredTokenException

Reproduction Steps

package main

import (
"context"
"fmt"
"log"

"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/credentials/stscreds"

)

func main() {
ctx := context.TODO()

// Replace with your parameters
roleARN := "arn:aws:iam::123456789012:role/YourRoleName"
sessionName := "your-session-name"
webIdentityTokenFile := "/path/to/your/web/identity/token"

// Load default AWS config
cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
	log.Fatalf("failed to load AWS SDK config: %v", err)
}

// Create a Web Identity Role Provider
roleProvider := stscreds.NewWebIdentityRoleProvider(cfg, roleARN, stscreds.IdentityTokenFile(webIdentityTokenFile), func(o *stscreds.WebIdentityRoleOptions) {
	o.RoleSessionName = sessionName
})

// Update the AWS config to use the Web Identity credentials provider
cfg.Credentials = aws.NewCredentialsCache(roleProvider)

// Create an S3 client
s3Client := s3.NewFromConfig(cfg)

// Example request: List S3 buckets
result, err := s3Client.ListBuckets(ctx, &s3.ListBucketsInput{})
if err != nil {
	log.Fatalf("failed to list S3 buckets: %v", err)
}

// Print bucket names
fmt.Println("Buckets:")
for _, bucket := range result.Buckets {
	fmt.Printf("%s\n", aws.ToString(bucket.Name))
}

}

Possible Solution

Add ExpiredTokenException to the retryable errors

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2 v1.32.6
github.com/aws/aws-sdk-go-v2/config v1.27.39
github.com/aws/aws-sdk-go-v2/credentials v1.17.47
github.com/aws/aws-sdk-go-v2/service/athena v1.46.2
github.com/aws/aws-sdk-go-v2/service/cloudtrail v1.43.3
github.com/aws/aws-sdk-go-v2/service/ec2 v1.188.0
github.com/aws/aws-sdk-go-v2/service/ecs v1.52.0
github.com/aws/aws-sdk-go-v2/service/eks v1.49.3
github.com/aws/aws-sdk-go-v2/service/iam v1.36.3
github.com/aws/aws-sdk-go-v2/service/identitystore v1.26.3
github.com/aws/aws-sdk-go-v2/service/kms v1.36.3
github.com/aws/aws-sdk-go-v2/service/lambda v1.62.1
github.com/aws/aws-sdk-go-v2/service/rds v1.85.2
github.com/aws/aws-sdk-go-v2/service/s3 v1.53.1
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.33.3
github.com/aws/aws-sdk-go-v2/service/sqs v1.37.1
github.com/aws/aws-sdk-go-v2/service/ssoadmin v1.28.3
github.com/aws/aws-sdk-go-v2/service/sts v1.33.2
github.com/aws/smithy-go v1.22.1
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.5 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.21 // indirect
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.6 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.5 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.7 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.6 // indirect

Compiler and Version used

go version go1.23.4 darwin/arm64

Operating System and version

MacOS 15.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions