Skip to content

[Storage] Add inactivity floor to qmdb::immutable Commit operation#3588

Merged
roberto-bayardo merged 9 commits into
mainfrom
danlaine/immutable-inactivity-floor
Apr 22, 2026
Merged

[Storage] Add inactivity floor to qmdb::immutable Commit operation#3588
roberto-bayardo merged 9 commits into
mainfrom
danlaine/immutable-inactivity-floor

Conversation

@danlaine
Copy link
Copy Markdown
Collaborator

@danlaine danlaine commented Apr 13, 2026

Summary

qmdb::immutable is an append-only authenticated database where each key is written exactly once. Applications use it for structures like transaction logs -- data that grows monotonically over the lifetime of the system.

The problem is that without a notion of an inactivity floor, the entire history must be retained and synced. In a network of replicas, a new node joining must fetch every operation from genesis to construct the current state. As the log grows, this becomes impractical. The database itself cannot determine which operations are "still needed" because, unlike qmdb::any where key overwrites make old operations provably dead, every immutable operation is alive by definition.

Only the application knows when old operations become irrelevant. For example, a transaction log might only need the last N blocks of history. This PR lets the application declare that knowledge by specifying an inactivity floor at commit time.

The floor is embedded directly in the Operation::Commit variant, making it part of the operation log and root digest. Every replica processing the same sequence of operations arrives at the same floor. This means:

  • Replicas agree on the floor without out-of-band coordination
  • The floor can be used as the lower bound of a sync target, so new nodes only fetch operations from the floor onward
  • Data below the floor can be pruned

Changes

Operation enum

Operation gains a Family generic parameter and the Commit variant now carries a Location<F> alongside its optional metadata:

// Before
enum Operation<K, V> {
    Set(K, V::Value),
    Commit(Option<V::Value>),
}

// After
enum Operation<F: Family, K, V> {
    Set(K, V::Value),
    Commit(Option<V::Value>, Location<F>),
}

The has_floor() method on the Operation trait now returns Some(loc) for Commit instead of None. Both the fixed-size codec (big-endian u64) and variable-size codec (varint) are updated to encode the new field.

Immutable struct

A new inactivity_floor_loc: Location<F> field is added to the Immutable struct. It is maintained through every state transition:

  • init_from_journal: Reads the floor from the last commit operation. The in-memory snapshot is replayed starting from the floor rather than from the journal's pruning boundary, so keys set before the floor are excluded from the snapshot.
  • apply_batch: Updates the floor from the batch. A monotonicity assertion (checked before any journal mutation) ensures the floor never decreases.
  • rewind: Restores the floor from the target commit operation.
  • from_sync_result: Extracts the floor from the last commit in the synced range and uses it as the snapshot replay start.
  • prune: Now checks against inactivity_floor_loc instead of last_commit_loc. The application can only prune up to what it has declared inactive.

A new inactivity_floor_loc() accessor is provided.

Batch API

merkleize() takes a new inactivity_floor: Location<F> parameter. The application passes this when building a batch to declare which operations are no longer needed. The floor is embedded in the resulting Commit operation and stored on MerkleizedBatch for use by apply_batch.

Partially resolves #3538

@danlaine danlaine self-assigned this Apr 13, 2026
@danlaine danlaine added this to Tracker Apr 13, 2026
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 13, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
commonware-mcp 37bbc72 Apr 21 2026, 10:24 PM

@danlaine danlaine added the breaking-format This PR modifies codec and/or storage formats. label Apr 13, 2026
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 13, 2026

Deploying monorepo with  Cloudflare Pages  Cloudflare Pages

Latest commit: 37bbc72
Status: ✅  Deploy successful!
Preview URL: https://35da9b51.monorepo-eu0.pages.dev
Branch Preview URL: https://danlaine-immutable-inactivit.monorepo-eu0.pages.dev

View logs

@danlaine danlaine moved this to In Progress in Tracker Apr 13, 2026
@danlaine danlaine force-pushed the danlaine/immutable-inactivity-floor branch from cc085d9 to ed7df57 Compare April 14, 2026 23:02
@danlaine danlaine marked this pull request as ready for review April 14, 2026 23:02
@danlaine danlaine moved this from In Progress to Ready for Review in Tracker Apr 14, 2026
@patrick-ogrady
Copy link
Copy Markdown
Contributor

As discussed, should we add this to keyless too?

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ea2c3da. Configure here.

Comment thread storage/src/qmdb/immutable/mod.rs
@roberto-bayardo roberto-bayardo added this pull request to the merge queue Apr 22, 2026
Merged via the queue into main with commit 2a7dd42 Apr 22, 2026
179 checks passed
@roberto-bayardo roberto-bayardo deleted the danlaine/immutable-inactivity-floor branch April 22, 2026 05:27
@github-project-automation github-project-automation Bot moved this from Ready for Review to Done in Tracker Apr 22, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

❌ Patch coverage is 97.90287% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.86%. Comparing base (505edcf) to head (37bbc72).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
storage/src/qmdb/immutable/mod.rs 98.00% 5 Missing and 9 partials ⚠️
storage/src/qmdb/immutable/operation/fixed.rs 68.75% 4 Missing and 1 partial ⚠️
@@           Coverage Diff            @@
##             main    #3588    +/-   ##
========================================
  Coverage   95.85%   95.86%            
========================================
  Files         441      442     +1     
  Lines      171201   172045   +844     
  Branches     4003     4013    +10     
========================================
+ Hits       164113   164933   +820     
- Misses       5828     5843    +15     
- Partials     1260     1269     +9     
Files with missing lines Coverage Δ
storage/src/qmdb/conformance.rs 100.00% <ø> (ø)
storage/src/qmdb/immutable/batch.rs 98.02% <100.00%> (+0.03%) ⬆️
storage/src/qmdb/immutable/fixed.rs 96.46% <100.00%> (+0.64%) ⬆️
storage/src/qmdb/immutable/operation/mod.rs 100.00% <100.00%> (ø)
storage/src/qmdb/immutable/operation/variable.rs 100.00% <100.00%> (ø)
storage/src/qmdb/immutable/sync/mod.rs 100.00% <100.00%> (ø)
storage/src/qmdb/immutable/variable.rs 96.57% <100.00%> (+0.59%) ⬆️
storage/src/qmdb/mod.rs 98.08% <ø> (-0.64%) ⬇️
storage/src/qmdb/sync/resolver.rs 46.60% <ø> (ø)
storage/src/qmdb/immutable/operation/fixed.rs 93.13% <68.75%> (+0.58%) ⬆️
... and 1 more

... and 9 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 505edcf...37bbc72. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-format This PR modifies codec and/or storage formats.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

[storage] Partial sync of keyless / immutable

3 participants