sccache + S3 on Jenkins (ECS Fargate): credentials expire after ~1 hour — best practices for long builds #2552
Unanswered
SSP17SCM51M
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi sccache maintainers/community,
We’re using sccache with S3 storage in Jenkins, and our builds often run longer than one hour. Around the 60-minute mark, S3 requests start failing due to expired AWS credentials. We’re looking for guidance on the best way to keep credentials valid for the full duration of long-running builds.
Environment:
sccache,Jenkins, AWS region,S3
Compute: ECS Fargate (Jenkins agents)
How credentials are provided: Jenkins Pipeline: AWS Steps withAWS sets temporary STS credentials in environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN).
Error after ~60 minutes: e.g., “ExpiredToken: The security token included in the request is expired”
What we’ve tried:
Increased the target IAM role’s MaxSessionDuration to 6 hours and requested a 3-hour session via the Jenkins plugin, but we still see expiry around 60 minutes.
Questions:
Does sccache (and the AWS SDK it uses) support automatically refreshing credentials during a long build when credentials are sourced from environment variables? Our understanding is that env-based STS credentials are static and won’t refresh.
If env-based credentials won’t refresh, what approaches are recommended in CI to avoid mid-build expiry?
Prefer a credentials source that supports auto-refresh (e.g., ECS Task Role/EC2 Instance Profile). On ECS/Fargate, the AWS SDK can auto-refresh container credentials from the task role.
If sccache starts with env-provided credentials that later expire, is there a supported way to have sccache re-read refreshed credentials without restarting the process?
Any known tips or configuration patterns for Jenkins + sccache + S3 (especially on ECS Fargate) that help with long-running builds?
We’re happy to test recommendations and report back. Thanks in advance for any guidance!
Beta Was this translation helpful? Give feedback.
All reactions