Skip to content

Retry failed JWT key refreshes with exponential backoff - #8226

Open
Amaury Chamayou (achamayou) with Copilot wants to merge 3 commits into
mainfrom
copilot/make-jwt-refresh-more-robust
Open

Retry failed JWT key refreshes with exponential backoff#8226
Amaury Chamayou (achamayou) with Copilot wants to merge 3 commits into
mainfrom
copilot/make-jwt-refresh-more-robust

Conversation

Copilot AI commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

JWT key refresh previously waited for the configured periodic interval after failures, which defaults to 30 minutes. Existing outage coverage checked failure metrics but not subsequent recovery.

Changes

  • Retry scheduling

    • Track retries independently per issuer.
    • Retry after min(5 seconds, key_refresh_interval).
    • Exponentially back off to the configured interval.
    • Coalesce pending retries and discard stale tasks safely.
    • Cancel retry state after success, issuer removal, or role change.
  • Outage recovery coverage

    • Extend the unavailable-endpoint test to start the issuer after an initial connection failure.
    • Verify keys refresh before the deliberately long periodic interval expires.
  • Documentation

    • Document retry behavior in the JWT operator guide and changelog.

Co-authored-by: achamayou <4016369+achamayou@users.noreply.github.com>
Copilot AI changed the title [WIP] Add retry logic for JWT refresh handling errors Retry failed JWT key refreshes with exponential backoff Aug 27, 2026
@achamayou
Amaury Chamayou (achamayou) marked this pull request as ready for review August 28, 2026 17:17
@achamayou
Amaury Chamayou (achamayou) requested a review from a team as a code owner August 28, 2026 17:17
Copilot AI lite review requested due to automatic review settings August 28, 2026 17:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves JWT/JWK auto-refresh robustness in CCF by adding per-issuer retry scheduling with exponential backoff, so transient OpenID/JWKS endpoint failures recover quickly rather than waiting for the (often long) periodic refresh interval.

Changes:

  • Add per-issuer retry state and delayed-task scheduling to retry failed refreshes quickly, then exponentially back off up to the configured refresh interval.
  • Extend the connection-failure e2e test to verify recovery once the OpenID endpoint becomes available.
  • Document the retry behavior in the JWT operator guide and add a changelog entry.

Custom instructions used:

  • .github/copilot-instructions.md
  • .github/instructions/reviewing.instructions.md
  • .github/instructions/changelog.instructions.md

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/node/jwt_key_auto_refresh.h Adds per-issuer retry scheduling with exponential backoff and cancellation on success/issuer removal/role change.
tests/jwt_test.py Extends the connection-failure test to start the issuer after an initial failure and assert refresh recovery.
doc/build_apps/auth/jwt.rst Documents the new retry + exponential backoff behavior.
CHANGELOG.md Records the user-visible behavior change for JWT auto-refresh retries.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/jwt_test.py
Comment on lines 457 to 461
with reserve_unlistened_local_port() as issuer_port:
issuer = infra.jwt_issuer.JwtIssuer(
f"https://{issuer_host}:{issuer_port}", cn=issuer_host
)
add_auto_refresh_jwt_issuer(network, primary, issuer, "jwt_connection_failure")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make JWT refresh more robust

3 participants