Skip to content

Build a resilient retry policy engine with exponential backoff and jitter #370

Description

@Lakes41

Difficulty: Advanced
Type: Feature
Recommended labels (if available in this repo): backend, performance, consistency, advanced

Background

GuildPass Core will eventually communicate with external infrastructure such as Stellar RPC endpoints, internal services and event processors. Temporary failures should not require every caller to implement its own retry loops.

This issue introduces a reusable retry primitive that can later be adopted by those integrations without depending on them today.

Problem

Naive retries can overload failing services, retry permanent errors unnecessarily or produce synchronized retry storms when many workers fail simultaneously.

Core needs explicit, deterministic retry semantics with configurable backoff and bounded execution.

Expected Outcome

Implement a generic asynchronous retry engine supporting exponential backoff, jitter, cancellation and caller-defined retry classification.

Suggested Implementation

The API should allow a caller to provide:

  • an async operation;
  • maximum attempts;
  • initial delay;
  • maximum delay;
  • backoff multiplier;
  • jitter configuration;
  • an optional AbortSignal;
  • a callback or predicate deciding whether an error is retryable.

The implementation should:

  • count the first execution as an attempt;
  • never retry beyond the configured maximum;
  • calculate exponential backoff safely;
  • cap delays at the configured maximum;
  • support deterministic jitter injection in tests through an injectable random source;
  • stop immediately when cancellation is requested;
  • avoid retrying errors classified as permanent;
  • preserve the final error when attempts are exhausted;
  • avoid timer leaks;
  • expose optional retry-attempt metadata to the caller;
  • avoid framework-specific dependencies.

Acceptance Criteria

  • Successful operations return immediately without unnecessary retries.
  • Retryable failures are retried up to the configured maximum.
  • Non-retryable failures are returned immediately.
  • Exponential backoff calculations are correct.
  • Maximum delay capping is enforced.
  • Jitter behaviour is configurable and testable deterministically.
  • AbortSignal cancellation stops future retries.
  • Timers are cleaned up correctly.
  • Invalid retry configuration is rejected.
  • The final thrown error preserves useful context.
  • Unit tests cover success, retry success, exhaustion, cancellation and non-retryable errors.
  • Tests do not rely on long real-time delays.
  • pnpm typecheck passes.
  • pnpm build passes.
  • pnpm test passes.

Likely Affected Files/Directories

packages/retry-policy/ or equivalent isolated package
unit tests

Independence Requirement

This issue must remain a generic resilience primitive. It must not depend on Stellar RPC, HTTP clients, Redis, Prisma or another contributor issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

GrantFox OSSGrantFox Open Source Sponsorship program tagMaybe RewardedIssue may qualify for a reward upon successful completion per campaign rulesThird CampaignOfficial FWC26 campaign issue — eligible for campaign scoring and rewardsadvancedAdvanced difficulty tasks requiring significant domain knowledge and implementation effortbackendBackend services, application logic, persistence integration, and server-side functionalityconsistencyPattern and convention standardization across the codebase for uniformityperformancePerformance optimization or latency/throughput improvement work

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions