Invalidate cached credentials on auth failures

[alexgromero]

Issue #, if available:
Python-9269

Overview

This updates credential refresh to invalidate cached refreshable credentials after service authentication failures.

When DEFAULT_NEW_CREDENTIAL_REFRESH is enabled, a service can reject otherwise unexpired credentials with ExpiredToken or InvalidToken, but botocore may still keep that credential set cached until its normal expiration time. This change marks the rejected credentials stale so the next request goes back through refresh instead of continuing to reuse the rejected credentials.

The legacy refresh path is unchanged.

What changed

On the signing side, RequestSigner now records the access key that signed the request in request.context. This is only done for the standard credential-chain case; request-scoped credentials and identity-cache flows are intentionally excluded.

On the response side, a new after-call handler watches for ExpiredToken and InvalidToken. When one of those errors is returned, it resolves the client's current credentials and invalidates them only if the cached access key still matches the access key that signed the rejected request. That avoids mutating fresh credentials when another thread has already completed a refresh.

The invalidation itself sets the cached credentials' expiry to now, so the next credential lookup routes through refresh. It also uses a non-blocking acquire on the refresh lock: if a refresh or another invalidation is already in progress, it returns immediately instead of blocking.

For providers with an intermediate cache layer, invalidating only the outer RefreshableCredentials object is not enough. AssumeRole, AssumeRoleWithWebIdentity, and SSO can otherwise refresh back into the same cached provider result. This change wires those providers to also expire the matching cached provider entry so the next refresh goes back to the underlying source.

Testing

Added coverage for:

• signer behavior that records signing_access_key and skips request-scoped / identity-cache cases
• auth-error invalidation event registration and handler behavior, including error_code_override
• refreshable credential invalidation when the access key matches or does not match
• provider-cache invalidation for AssumeRole, AssumeRoleWithWebIdentity, and SSO
• invalidation preserving the existing refresh backoff flow rather than bypassing it
• strict refreshable credentials still invalidating while preserving fail-fast refresh behavior when refresh itself fails

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 93.10345% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (feature/credential-refresh@0056fae). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
botocore/credentials.py	86.20%	4 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@                      Coverage Diff                      @@
##             feature/credential-refresh    #3777   +/-   ##
=============================================================
  Coverage                              ?   92.57%           
=============================================================
  Files                                 ?       68           
  Lines                                 ?    16117           
  Branches                              ?        0           
=============================================================
  Hits                                  ?    14920           
  Misses                                ?     1197           
  Partials                              ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ubaskota]

It looks good to me as is. I've added a comment to point out a test gap and will approve it after its addressed. Thanks

[ubaskota]

Can we add a test for the case where the refresh lock is already held when _invalidate is called?

[alexgromero]

Yep, added in ae17afe.