Skip to content

[storage/qmdb/bench] chained-growth benchmark#3633

Merged
patrick-ogrady merged 1 commit into
mainfrom
chained-growth-bench
Apr 20, 2026
Merged

[storage/qmdb/bench] chained-growth benchmark#3633
patrick-ogrady merged 1 commit into
mainfrom
chained-growth-bench

Conversation

@roberto-bayardo
Copy link
Copy Markdown
Collaborator

@roberto-bayardo roberto-bayardo commented Apr 20, 2026

Summary

Add a chained_growth benchmark to the qmdb suite that exercises the pipelined-speculation workload: applying a parent batch while a child batch is still alive, repeatedly, to exercise Current QMDB's committed-bitmap parent chain.

Benchmark shape

  1. Setup (untimed): seed NUM_KEYS=1_000_000 keys, then grow a chain of PREBUILT_CHAIN=10_000 batches applying each parent while the child is alive. Commit and sync, so buffered seed data doesn't leak into the timed region's fsync.
  2. Timed: do batches=100 more merkleize + apply iterations on top of the pre-built chain, then drop the final parent.

Each batch contains a single random key update (UPDATES_PER_BATCH=1) so that each batch's chunk overlay covers a tiny fraction of the total chunks. Combined with the deep pre-built chain, this forces each chunk read during merkleize to walk the chain before finding a matching layer — exposing the asymptotic cost of chain growth that shallower-coverage workloads mask.

Only Current variants are exercised (the chain-growth behavior is specific to Current's bitmap batching) with chunk sizes of 32 & 256. Variants are limited to fixed and MMB since value type and merkle family aren't relevant for this bench.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 20, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
commonware-mcp aa59e3e Apr 20 2026, 07:26 PM

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 20, 2026

Deploying monorepo with  Cloudflare Pages  Cloudflare Pages

Latest commit: aa59e3e
Status: ✅  Deploy successful!
Preview URL: https://a4bf433c.monorepo-eu0.pages.dev
Branch Preview URL: https://chained-growth-bench.monorepo-eu0.pages.dev

View logs

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new QMDB benchmark focused on “chained-growth” merkleization behavior in Current variants, intended to stress the committed-bitmap parent-chain walk cost under a deep chain with very sparse overlays.

Changes:

  • Introduces chained_growth Criterion benchmark that seeds a large DB, pre-builds a deep speculative/commit chain, then measures additional merkleize+apply iterations.
  • Adds the new benchmark module to the QMDB benches entrypoint so it runs as part of the suite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
storage/src/qmdb/benches/chained_growth.rs New chained-growth benchmark implementation for Current fixed+MMB variants at chunk sizes 32 and 256.
storage/src/qmdb/benches/bench.rs Registers the new chained_growth benchmark group in criterion_main!.

Comment thread storage/src/qmdb/benches/chained_growth.rs Outdated
Comment thread storage/src/qmdb/benches/chained_growth.rs
Comment thread storage/src/qmdb/benches/chained_growth.rs
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread storage/src/qmdb/benches/chained_growth.rs
Comment thread storage/src/qmdb/benches/chained_growth.rs Outdated
@roberto-bayardo roberto-bayardo force-pushed the chained-growth-bench branch 4 times, most recently from 418ee05 to f17b653 Compare April 20, 2026 19:15
@roberto-bayardo roberto-bayardo requested a review from Copilot April 20, 2026 19:15
@roberto-bayardo roberto-bayardo marked this pull request as ready for review April 20, 2026 19:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@roberto-bayardo roberto-bayardo added this pull request to the merge queue Apr 20, 2026
@patrick-ogrady patrick-ogrady removed this pull request from the merge queue due to a manual request Apr 20, 2026
@patrick-ogrady patrick-ogrady merged commit a1c0971 into main Apr 20, 2026
178 of 179 checks passed
@patrick-ogrady patrick-ogrady deleted the chained-growth-bench branch April 20, 2026 22:50
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.81%. Comparing base (c6a805b) to head (aa59e3e).
⚠️ Report is 3 commits behind head on main.

@@            Coverage Diff             @@
##             main    #3633      +/-   ##
==========================================
- Coverage   95.81%   95.81%   -0.01%     
==========================================
  Files         440      440              
  Lines      167922   167922              
  Branches     3922     3922              
==========================================
- Hits       160900   160891       -9     
- Misses       5800     5806       +6     
- Partials     1222     1225       +3     

see 3 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6a805b...aa59e3e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants