[storage] use Blob::write_at_sync#3858
Merged
Merged
Conversation
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
commonware-mcp | 80c2480 | May 22 2026, 08:51 PM |
Benchmark resultsTip ✅ PASSED: No benchmark exceeded the regression threshold. Benchmark comparison table
Baseline commit(s): |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates storage primitives and runtime buffering layers to use per-write durability helpers (Blob::write_at_sync, Metadata::put_sync, Archive::put_sync, Cache::put_sync) where a single write can be made durable without requiring a separate full sync(). This reduces unnecessary durability barriers while preserving correct behavior for cases that still require a full sync (notably shrink/resizes and multi-write flushes).
Changes:
- Update
Metadatafull-rewrite sync logic to useBlob::write_at_syncfor non-shrinking rewrites, while retainingwrite_at + resize + syncfor shrinking rewrites. - Replace multiple
put/write_at+synccall pairs across storage code (including tests and fuzz target corruption steps) with the corresponding*_synchelpers. - Enhance runtime buffer implementations (
buffer::Writeandbuffer::paged::Append) with explicit “needs full sync” tracking to safely usewrite_at_synconly when no prior unsynced mutation exists; add targeted tests validating durability-barrier behavior.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| storage/src/qmdb/keyless/compact.rs | Use Metadata::put_sync in tests instead of put + sync. |
| storage/src/qmdb/immutable/compact.rs | Use Metadata::put_sync in tests instead of put + sync. |
| storage/src/ordinal/mod.rs | Use Blob::write_at_sync in corruption tests to persist single writes without extra sync. |
| storage/src/metadata/storage.rs | Optimize full rewrites: non-shrinking rewrites use write_at_sync; shrinking rewrites still require resize + full sync. |
| storage/src/metadata/mod.rs | Update tests to use write_at_sync / put_sync helpers for clearer durable writes. |
| storage/src/merkle/persisted/full.rs | Replace metadata put + sync with put_sync where equivalent; update test accordingly. |
| storage/src/merkle/persisted/compact.rs | Replace metadata put + sync with put_sync in implementation and tests. |
| storage/src/journal/segmented/variable.rs | Use write_at_sync in corruption tests; adjust one test payload to force a physical resize path. |
| storage/src/journal/segmented/oversized.rs | Use write_at_sync in corruption tests to persist single writes without extra sync. |
| storage/src/journal/contiguous/fixed.rs | Use put_sync for durable metadata writes; ensure remove path still performs metadata.sync(). |
| storage/src/freezer/storage.rs | Use write_at_sync in corruption test for a single durable write. |
| storage/src/freezer/mod.rs | Use write_at_sync in corruption tests to persist single writes without extra sync. |
| storage/src/archive/immutable/storage.rs | Publish checkpoint via metadata.put_sync (single durable op) after underlying state is durable. |
| storage/src/archive/immutable/mod.rs | Use Archive::put_sync in test instead of put + sync. |
| storage/fuzz/fuzz_targets/oversized_recovery.rs | Persist fuzz corruption writes with write_at_sync instead of write_at + sync. |
| runtime/src/utils/buffer/write.rs | Introduce shared writer State tracking prior unsynced mutations to safely use write_at_sync on single-write flushes. |
| runtime/src/utils/buffer/paged/append.rs | Add needs_sync tracking and conditional write_at_sync usage; refine shrink behavior; add durability-focused tests. |
| runtime/src/utils/buffer/mod.rs | Add SyncTrackingBlob test helper and new tests asserting correct range-sync vs full-sync selection. |
# Conflicts: # runtime/src/utils/buffer/mod.rs # runtime/src/utils/buffer/paged/append.rs # runtime/src/utils/buffer/write.rs # storage/src/journal/segmented/variable.rs
Deploying monorepo with
|
| Latest commit: |
80c2480
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://96576614.monorepo-eu0.pages.dev |
| Branch Preview URL: | https://andre-storage-write-at-sync.monorepo-eu0.pages.dev |
Contributor
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 619c771. Configure here.
patrick-ogrady
approved these changes
May 22, 2026
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## main #3858 +/- ##
==========================================
- Coverage 95.77% 95.77% -0.01%
==========================================
Files 486 486
Lines 201251 201235 -16
Branches 4876 4876
==========================================
- Hits 192754 192730 -24
- Misses 6862 6866 +4
- Partials 1635 1639 +4
... and 6 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR updates
Metadatafull rewrites to useBlob::write_at_syncwhen the new metadata payload does not shrink the blob. A non-shrinking rewrite is a single durable write, so it no longer needs a separate sync. Shrinking rewrites still usewrite_at + resize + syncbecause the resize must also be made durable.The rest of the changes replace manual
put/write_atfollowed bysyncwith the corresponding sync helper where that is equivalent and clearer. This includes uses ofMetadata::put_sync,Archive::put_sync,Cache::put_sync, and directBlob::write_at_syncin tests.Most storage primitives already benefit from
write_at_syncindirectly through the runtime buffering layers.buffer::Writeandbuffer::paged::Appendcan usewrite_at_syncwhen flushing a single write with no earlier dirty mutation, so databases built on journals, ordinal storage, cache/archive paths, and QMDB logs inherit that behavior.Depends on #3857.