Skip to content

feat: adaptive V2 reconstruction with split/grouped range strategy#692

Draft
XciD wants to merge 1 commit intofeat/v2-reconstruction-multirangefrom
feat/v2-reconstruction-adaptive
Draft

feat: adaptive V2 reconstruction with split/grouped range strategy#692
XciD wants to merge 1 commit intofeat/v2-reconstruction-multirangefrom
feat/v2-reconstruction-adaptive

Conversation

@XciD
Copy link
Member

@XciD XciD commented Mar 10, 2026

Summary

  • Split large ranges (avg >= 256KB, <= 200 ranges) into individual single-range HTTP requests for better CDN cache hit rates and parallelism
  • Keep small/fragmented ranges grouped in multi-range requests to avoid per-request overhead (~15ms per connection)
  • Includes [BENCH] instrumentation (eprintln) for V1 vs V2 comparison: reconstruction timing, S3 response details, multipart parsing

This is the "V2-adaptive" approach from the benchmark results:

  label                  r/xorb |      V1 V2-orig V2-adpt |
 ----------------------------------------------------------
  gpt2_548MB                1.0 |    1.8s    2.0s    1.9s |
  smollm17b_3.4GB           1.0 |    6.0s    5.7s    4.2s |
  qwen35b_gguf_69GB        40.9 |  101.7s  301.2s   72.1s |

V2-adaptive is on par with V1 for non-fragmented files and 29% faster on highly fragmented ones.

Split large ranges (avg >= 256KB) into individual single-range HTTP requests
for better CDN cache hit rates and parallelism. Keep small/fragmented ranges
grouped in multi-range requests to avoid per-request overhead.

Includes benchmark instrumentation for V1 vs V2 comparison.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant