Skip to content

feat(layer): [1/N] implement basic request hedging for stats and read#7408

Open
dentiny wants to merge 2 commits intoapache:mainfrom
dentiny:hjiang/request-hedging-layer-1
Open

feat(layer): [1/N] implement basic request hedging for stats and read#7408
dentiny wants to merge 2 commits intoapache:mainfrom
dentiny:hjiang/request-hedging-layer-1

Conversation

@dentiny
Copy link
Copy Markdown
Contributor

@dentiny dentiny commented Apr 18, 2026

Which issue does this PR close?

Closes #7189

Rationale for this change

As described in the attached issue, request hedging is a useful to cut down tail latency.

What changes are included in this PR?

Similar to retry, the biggest caveat is operation idempotency, so in this PR to focus on the most important thing, I only implemented read + stats.

A few considerations and design choices:

  • No max timeout or max attempt
    • opendal takes a layering design with each layer taking its own responsibility; timeout layer should be used together to cap unbounded hedging
  • Request hedging returns at the first completed request, instead of first succeeded request
    • Similar to the reason above, retry failed attempts is achieved by retry layer
  • Request hedginig happens at operation layer, same design choice as retry layer
    • For concurrent read, hedged request take place for each chunk read
  • Enabled operation and delay
    • When users use hedged request, all operations (which has implemented hedged request) are default on, but users are also allowed to opt off for certain operations
    • A default hedging delay is provided, meanwhile users are able to change the fallback/default delay for all operations, and configure delay for one specific operation
  • Unit tests are all built upon timeouts, to reduce flaky test, server side injected delay and hedged request delay are kept 10x difference

Are there any user-facing changes?

Yes, it adds a new layer users could leverage.

AI Usage Statement

opus 4.6 made most of the code change and I did the code review.

@dentiny dentiny requested a review from Xuanwo as a code owner April 18, 2026 07:07
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Apr 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: hedged request support for read-only operations

1 participant