CORTEX

COordinated Retry Throttling and EXecution

An adaptive retry storm prevention engine built for Amazon-scale distributed systems.

The Problem

Amazon processes ~4,000 orders per minute. Each order triggers retries across payment gateways, inventory services, and shipping APIs. When a downstream service starts failing, every client retries independently -- 3 retries per failure means traffic amplifies from 10,000 to 60,000 requests per second. The already-struggling service gets crushed under 6x load, dies completely, and when it recovers, the backlog of pending retries floods it again -- killing it a second time. This is a retry storm.

These aren't theoretical. The October 2025 DynamoDB outage followed exactly this pattern: service recovers, retry backlog crashes it, repeat indefinitely.

CORTEX prevents all of this with a single function call.

The Core Idea

"One score drives every retry decision."

CORTEX computes a Composite Pressure Score (CPS) -- a single number from 0.0 to 1.0 -- that unifies 5 independent signals into ONE decision function:

Full health data   ->  CPS-driven unified decisions (optimal)
Partial signals    ->  Graceful degradation with available data
CPS computation    ->  Standard independent checks (fallback)
fails entirely

When all signals are available, CORTEX uses adaptive weights and priority-based shedding for precise control. When the CPS engine itself fails, it falls back to standard independent checks instead of crashing. Your service never stops.

Same engine. Same API. Same code path. System pressure drives behavior.

Why This Exists

Amazon's current retry approach uses 4 independent mechanisms that contradict each other under pressure:

Exponential backoff delays retries, but doesn't reduce their count
Rate limiters cap throughput, but don't know about circuit state
Circuit breakers trip on failure rate, but don't see latency warnings
Retry budgets limit total retries, but don't consider request priority

CORTEX replaces all four with a unified score. The CPS formula borrows from real distributed systems:

Real-World System	Concept Borrowed	What It Does in CORTEX
Amazon SDK	Exponential Backoff	Base delay strategy, enhanced with CPS modulation
Netflix Hystrix	Circuit Breaking	3-state isolation with probe-based recovery
Google SRE	Error Budget	Self-regulating token bucket with success-refill
TCP Congestion	AIMD Control	Additive increase, multiplicative decrease via CPS
Control Theory	PID-like Feedback	5-signal weighted sum with adaptive tuning

CORTEX's innovation is computing one number (CPS) that makes all retry components work together instead of independently.

Architecture

                        +----------------------+
                        |   Upstream Services   |
                        |  (Payment, Inventory, |
                        |   Shipping, Search)   |
                        +----------+-----------+
                                   v
              +----------------------------------------+
              |            CORTEX API Layer             |
              |                                        |
              |   cortex.shouldRetry(pri, att, retry)   |
              |   cortex.recordSuccess(latencyMs)       |
              |   cortex.recordFailure(latencyMs)       |
              +------------------+---------------------+
                                 v
              +----------------------------------------+
              |           CPS Engine (Brain)            |
              |                                        |
              |   CPS = w1*F + w2*L + w3*Q + w4*B + w5*C|
              |                                        |
              |   +----------+  +-------------------+  |
              |   | Adaptive |  | Priority Manager  |  |
              |   | Weights  |  | (load shedding)   |  |
              |   +----------+  +-------------------+  |
              |   +------------------+ +------------+  |
              |   | Adaptive Backoff | | Recovery   |  |
              |   | (CPS-modulated)  | | Manager    |  |
              |   +------------------+ +------------+  |
              +------------------+---------------------+
                                 v
              +----------------------------------------+
              |         Signal Collection Layer         |
              |                                        |
              |  +------------+ +-------------------+  |
              |  | Signal F   | | Signal L          |  |
              |  | Failure    | | Latency Deviation |  |
              |  | Rate       | | (earliest warning)|  |
              |  +------------+ +-------------------+  |
              |  +------------+ +-------------------+  |
              |  | Signal Q   | | Signal B          |  |
              |  | Queue      | | Budget Burn       |  |
              |  | Pressure   | | Rate              |  |
              |  +------------+ +-------------------+  |
              |  +-------------------+                  |
              |  | Signal C          |                  |
              |  | Circuit State     |                  |
              |  +-------------------+                  |
              +------------------+---------------------+
                                 v
              +----------------------------------------+
              |        Resilience Primitives            |
              |                                        |
              |  +------------------+ +-------------+  |
              |  |  SlidingWindow   | | TokenBucket  |  |
              |  |  (F + L signals) | | (B signal)   |  |
              |  +------------------+ +-------------+  |
              |  +------------------+                   |
              |  | CircuitBreaker   |                   |
              |  | (C signal)       |                   |
              |  +------------------+                   |
              +----------------------------------------+

The CPS Decision Pipeline

Every retry request passes through exactly 5 checks. Each check uses the unified CPS score. If CPS computation fails at any point, the system falls back to independent checks (Amazon's standard approach).

+-----------------------------------------------------------------+
|                     Failed request arrives                       |
+--------------------------+--------------------------------------+
                           v
+------------------------------------------------------------------+
|  CHECK 1: RETRYABLE?                                             |
|  Is this error retryable? (4xx -> NO, 5xx -> YES)                 |
|  NO  -> NOT_RETRYABLE (return error to caller)                   |
|  YES -> continue                                                  |
+--------------------------+---------------------------------------+
                           v
+------------------------------------------------------------------+
|  CHECK 2: MAX RETRIES?                                           |
|  Has this request exceeded max retry attempts?                   |
|  YES -> MAX_RETRIES_EXCEEDED (give up)                           |
|  NO  -> continue                                                  |
+--------------------------+---------------------------------------+
                           v
+------------------------------------------------------------------+
|  CHECK 3: COMPUTE CPS                                            |
|  CPS = w1*F + w2*L + w3*Q + w4*B + w5*C                        |
|  CPS >= 0.75 -> EMERGENCY: force circuit OPEN, reject all       |
|  Circuit already OPEN -> CIRCUIT_OPEN: fail fast                 |
|  CPS < 0.75 -> continue                                          |
+--------------------------+---------------------------------------+
                           v
+------------------------------------------------------------------+
|  CHECK 4: PRIORITY + BUDGET                                      |
|  Is priority >= floor(CPS * 10)? (priority-based shedding)      |
|  NO  -> REJECT (priority too low for current pressure)           |
|  Does token bucket have tokens?                                  |
|  NO  -> REJECT (retry budget exhausted)                          |
|  YES -> continue                                                  |
+--------------------------+---------------------------------------+
                           v
+------------------------------------------------------------------+
|  CHECK 5: COMPUTE DELAY                                          |
|  delay = random(0, base * 2^attempt * (1 + CPS * 10))           |
|  -> RETRY with calculated delay                                  |
+------------------------------------------------------------------+

Quick Start

Prerequisites

Java 21 or higher
No external dependencies

Build & Test

# Compile everything
javac -d out --release 21 $(find src -name "*.java")

# Run the simulation (all 4 failure scenarios)
java -cp out cortex.SimulationRunner

# Compile and run tests
javac -d out --release 21 $(find src tests -name "*.java")
java -cp out tests.SlidingWindowTest
java -cp out tests.CircuitBreakerTest
java -cp out tests.CPSEngineTest
java -cp out tests.AdaptiveBackoffTest
java -cp out tests.RecoveryFlowTest

Your First Retry Decision (3 Lines of Code)

import cortex.api.Cortex;
import cortex.api.RetryDecision;

// Create engine with sensible defaults
Cortex cortex = Cortex.withDefaults();

// A payment request failed -- should we retry?
RetryDecision decision = cortex.shouldRetry(
    9,       // priority (9 = PAYMENT, highest)
    0,       // attempt number (first retry)
    true     // is this error retryable?
);

if (decision.shouldRetry()) {
    Thread.sleep(decision.getDelayMs());
    // retry the request
}

System.out.println(decision);
// RetryDecision[RETRY, delay=523ms, CPS=0.0400, state=HEALTHY, priority=9/0]

That's it. One function. Three arguments. The config builder lets you customize every parameter.

How to Use CORTEX

The Builder Pattern

Every CORTEX instance is configured with a fluent builder:

Cortex cortex = Cortex.withConfig(
    CortexConfig.builder()
        .tokenBucketCapacity(500)            // Retry budget size
        .tokenRefillRate(100.0)              // Tokens per second
        .tokenSuccessBonus(5)                // Bonus tokens per success
        .circuitFailureThreshold(0.60)       // 60% failure -> trip circuit
        .circuitSustainedDuration(10_000)    // Must sustain for 10 seconds
        .circuitCooldown(15_000)             // Wait 15s before probing
        .backoffBaseDelay(1000)              // 1 second base delay
        .backoffMaxDelay(30_000)             // 30 second max delay
        .maxRetryAttempts(3)                 // Max 3 retry attempts
        .adaptiveWeights(true)               // Enable self-tuning weights
        .deterministicMode(false)            // Real randomness (not testing)
        .build()
);

The 3-Method API

CORTEX has exactly 3 public methods. Your application code stays minimal:

Method	When to Call	What It Does
`shouldRetry(priority, attempt, retryable)`	After a request fails	Returns a `RetryDecision` with action + delay
`recordSuccess(latencyMs)`	After a request succeeds	Updates all 5 signals positively
`recordFailure(latencyMs)`	After a request fails	Updates all 5 signals negatively

Reading the Decision

RetryDecision decision = cortex.shouldRetry(priority, attempt, true);

if (decision.shouldRetry()) {
    Thread.sleep(decision.getDelayMs());
    System.out.println("Retrying after " + decision.getDelayMs() + "ms");
    System.out.println("System state: " + decision.getSystemState());
    System.out.println("CPS: " + decision.getCps());
} else {
    System.out.println("Rejected: " + decision.getReason());
}

All Possible Outcomes

Action	Meaning	What to Do
`RETRY`	Retry is allowed with calculated delay	Wait the delay, then retry
`REJECT`	Priority too low or budget exhausted	Return error to caller
`CIRCUIT_OPEN`	Service is dead, fail fast	Return error immediately
`NOT_RETRYABLE`	Error is not retryable (4xx)	Return error to caller
`MAX_RETRIES_EXCEEDED`	Too many attempts	Dead letter or return error

Real-World Scenarios

1. Payment Service Under Load

Cortex cortex = Cortex.withDefaults();

// Normal traffic -- all retries allowed
for (int i = 0; i < 100; i++) {
    cortex.recordSuccess(40);  // 40ms latency, healthy
}

RetryDecision d = cortex.shouldRetry(9, 0, true);
// RETRY -- CPS is low, payment priority (9) easily clears threshold

// Service starts failing
for (int i = 0; i < 200; i++) {
    cortex.recordFailure(5000);  // 5 second timeouts
}

// Low-priority retry blocked, high-priority still allowed
cortex.shouldRetry(1, 0, true);  // REJECT -- analytics (pri=1) shed
cortex.shouldRetry(9, 0, true);  // RETRY  -- payment (pri=9) survives

2. Priority-Based Load Shedding

// As CPS rises, lower priorities are shed first:
//   CPS 0.30 -> Analytics (1), Recommendations (2) dropped
//   CPS 0.50 -> Search (4), Content (5) also dropped
//   CPS 0.70 -> Only Payment (9) and Health Checks (10) survive
//   CPS 0.90 -> Almost everything blocked

// In code:
int minPriority = (int) Math.floor(cps * 10);
// Only requests with priority >= minPriority are allowed

3. Recovery Storm Prevention

// The DynamoDB October 2025 pattern:
//   Service crashes -> all clients accumulate retries
//   Service recovers -> retries flood it -> crashes again -> repeat

// CORTEX prevents this because:
//   Service recovers -> CPS is still elevated (sliding window memory)
//   -> only high-priority traffic allowed -> gradual ramp
//   -> CPS decays naturally as window clears -> smooth recovery

4. Fallback Guarantee

// If CPS computation throws ANY exception:
try {
    return computeUnifiedDecision(priority, attempt);  // CPS-based
} catch (Exception e) {
    return computeFallbackDecision(priority, attempt, e);  // Independent checks
}

// Fallback uses circuit breaker + token bucket independently
// CORTEX's worst case = Amazon's current normal case

5. Deterministic Testing

// Enable deterministic mode for reproducible tests
Cortex cortex = Cortex.withConfig(
    CortexConfig.builder()
        .deterministicMode(true)  // All randomness returns midpoint
        .build()
);

// Same inputs always produce same outputs -- no flaky tests
long delay1 = cortex.shouldRetry(5, 1, true).getDelayMs();
long delay2 = cortex.shouldRetry(5, 1, true).getDelayMs();
assert delay1 == delay2;  // Always true in deterministic mode

System States

CPS Range	State	Behavior
0.00 - 0.25	`HEALTHY`	All retries allowed, normal delays
0.25 - 0.50	`DEGRADED`	Low-priority retries shed, delays stretched
0.50 - 0.75	`CRITICAL`	Most retries blocked, long delays
0.75 - 1.00	`EMERGENCY`	Circuit forced OPEN, all retries blocked

Benchmark Results

Scenario	Naive Success	CORTEX Success	Traffic Amp	Recovery
Black Friday (Payment DB)	8.5%	67.0%	6.0x -> 1.32x	NEVER -> 35s
DNS Crash (DynamoDB pattern)	8.8%	71.0%	6.0x -> 1.38x	NEVER -> 18s
Food Delivery (Dinner Rush)	11.7%	75.3%	6.0x -> 1.32x	DEAD -> Survived
UPI Salary Day (Bank API)	12.3%	71.2%	6.0x -> 1.25x	DEAD -> Survived

Key Numbers

Metric	Without CORTEX	With CORTEX	Improvement
Peak Traffic Amplification	6.00x	1.32x	78% reduction
Unnecessary Retries	2,520,329	48,498	98% fewer
Overall Success Rate	8.5%	67.0%	7.9x higher
Recovery Time	NEVER	35 seconds	System survives

Run the Simulations

# All 4 failure scenarios with side-by-side comparison
java -cp out cortex.SimulationRunner

Output includes second-by-second timeline, color-coded CPS tracking, and per-scenario comparison metrics.

Test Suite

163 tests. All passing. Zero failures.

Category	Tests	What's Covered
Unit -- SlidingWindow	11	Initial state, failure rate, success rate, mixed traffic, P99 latency
Unit -- CircuitBreaker	114	Initial state, closed operation, sustained trip, half-open probe, force-open, state values
Unit -- CPSEngine	23	All 4 states, max/min CPS, priority thresholds, weight bounds, normalization
Unit -- AdaptiveBackoff	7	Deterministic mode, attempt scaling, CPS scaling, delay cap, fallback
Integration -- RecoveryFlow	8	Full lifecycle, fallback guarantee, max retries, non-retryable, priority shedding

Project Structure

adaptive-retry-orchestrator/
|
+-- docs/                                      # Engineering documentation
|   +-- architecture/                          #   System design docs
|   |   +-- system_design.md                   #     Layer architecture + failure modes
|   |   +-- retry_lifecycle.md                 #     Complete request flow
|   |   +-- cascading_failure_analysis.md      #     Retry storm anatomy
|   |   +-- adaptive_backoff_strategy.md       #     CPS-modulated delay design
|   |   +-- distributed_coordination.md        #     Why local-only computation
|   |   +-- circuit_breaking_design.md         #     3-state model + CPS integration
|   |
|   +-- algorithms/                            #   Algorithm deep dives
|   |   +-- adaptive_retry_scoring.md          #     CPS formula + weight tuning
|   |   +-- exponential_backoff.md             #     Growth pattern + cap
|   |   +-- jitter_algorithms.md               #     Full vs equal vs decorrelated
|   |   +-- retry_budgeting.md                 #     Self-regulating budget
|   |   +-- token_bucket_throttling.md         #     Refill mechanics
|   |   +-- queue_backpressure.md              #     Feedback loop design
|   |   +-- cascading_failure_prevention.md    #     Multi-layer protection
|   |
|   +-- benchmarks/                            #   Performance analysis
|       +-- throughput_results.md              #     All scenario results
|       +-- retry_storm_simulation.md          #     Storm phases + prevention
|       +-- latency_analysis.md               #     Early warning signal analysis
|
+-- src/cortex/
|   +-- api/                                   #  Public SDK Layer
|   |   +-- Cortex.java                        #    3-method API entry point
|   |   +-- CortexConfig.java                  #    Builder-pattern configuration
|   |   +-- RetryDecision.java                 #    Decision output with observability
|   |
|   +-- core/                                  #  Intelligence Layer
|   |   +-- CPSEngine.java                     #    CPS computation + adaptive weights
|   |   +-- AdaptiveBackoff.java               #    CPS-modulated exponential backoff
|   |   +-- PriorityManager.java               #    CPS-driven load shedding
|   |   +-- RecoveryManager.java               #    Recovery storm prevention
|   |
|   +-- resilience/                            #  Resilience Primitives
|   |   +-- CircuitBreaker.java                #    3-state failure isolation
|   |   +-- TokenBucket.java                   #    Self-regulating retry budget
|   |   +-- SlidingWindow.java                 #    Rolling window metrics
|   |
|   +-- metrics/                               #  Signal Collection
|   |   +-- FailureTracker.java                #    Signal F: failure rate
|   |   +-- LatencyTracker.java                #    Signal L: latency deviation
|   |   +-- QueueTracker.java                  #    Signal Q: queue pressure
|   |   +-- MetricsSnapshot.java               #    Point-in-time capture
|   |
|   +-- model/                                 #  Domain Models
|   |   +-- CPSState.java                      #    HEALTHY / DEGRADED / CRITICAL / EMERGENCY
|   |   +-- RequestPriority.java               #    Priority enum with auto-detection
|   |   +-- RetryContext.java                  #    Immutable retry context
|   |   +-- ServiceHealth.java                 #    Aggregated service health
|   |
|   +-- simulation/                            #  Failure Scenarios
|   |   +-- FailureSimulator.java              #    Core simulation engine
|   |   +-- BlackFridayScenario.java           #    Payment DB crash under peak load
|   |   +-- RecoveryStormScenario.java         #    DNS failure (DynamoDB Oct 2025)
|   |   +-- FoodDeliveryScenario.java          #    Dinner rush DB crash
|   |   +-- UpiSurgeScenario.java              #    Salary day bank API overload
|   |
|   +-- utils/                                 #  Shared Utilities
|   |   +-- TimeUtils.java                     #    Deterministic time for testing
|   |   +-- RandomUtils.java                   #    Deterministic randomness
|   |   +-- ConsoleLogger.java                 #    Color-coded terminal output
|   |
|   +-- SimulationRunner.java                  #  Entry point: runs all scenarios
|
+-- tests/                                     #  Test Suites
|   +-- SlidingWindowTest.java                 #    11 tests: failure/success/latency
|   +-- CircuitBreakerTest.java                #    114 tests: states/transitions/probes
|   +-- CPSEngineTest.java                     #    23 tests: formula/states/weights
|   +-- AdaptiveBackoffTest.java               #    7 tests: delays/caps/determinism
|   +-- RecoveryFlowTest.java                  #    8 tests: full lifecycle integration
|
+-- LICENSE                                    #  MIT License

Inspired By

System	What CORTEX Borrows
Amazon SDK	Exponential backoff with full jitter as the base delay strategy
Netflix Hystrix	3-state circuit breaker with probe-based recovery
Google SRE	Error budget concept adapted into self-regulating token bucket
TCP Congestion Control	Additive increase / multiplicative decrease through CPS modulation
Control Theory	PID-like feedback loop: 5 weighted signals with adaptive tuning
AWS Builder's Library	Retry storm prevention patterns and cascading failure analysis

CORTEX's contribution: computing a single Composite Pressure Score that makes all retry components work together through one unified decision function -- so retry logic, circuit breaking, rate limiting, backoff, and priority shedding cooperate instead of contradicting each other.

License

MIT -- see LICENSE

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
.github/workflows		.github/workflows
infra		infra
src/cortex		src/cortex
tests		tests
.gitignore		.gitignore
ARCHITECTURE.md		ARCHITECTURE.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pom.xml		pom.xml

Folders and files

Latest commit

History

Repository files navigation

CORTEX

COordinated Retry Throttling and EXecution

The Problem

The Core Idea

Why This Exists

Architecture

The CPS Decision Pipeline

Quick Start

Prerequisites

Build & Test

Your First Retry Decision (3 Lines of Code)

How to Use CORTEX

The Builder Pattern

The 3-Method API

Reading the Decision

All Possible Outcomes

Real-World Scenarios

1. Payment Service Under Load

2. Priority-Based Load Shedding

3. Recovery Storm Prevention

4. Fallback Guarantee

5. Deterministic Testing

System States

Benchmark Results

Key Numbers

Run the Simulations

Test Suite

Project Structure

Inspired By

License

About

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages