craft: S5 RAFT entry apply — SyncRSCommitLSN + InternalLogin (SDSTOR-22886, SDSTOR-22887) - #176
Draft
sbinmalek wants to merge 8 commits into
Draft
craft: S5 RAFT entry apply — SyncRSCommitLSN + InternalLogin (SDSTOR-22886, SDSTOR-22887)#176sbinmalek wants to merge 8 commits into
sbinmalek wants to merge 8 commits into
Conversation
Implement the apply side of the SyncRSCommitLSN RAFT entry: on_commit
now parses the entry header/key and dispatches to
apply_sync_rs_commit_lsn, which reconciles empty_slots, catches up
missing journal data from a peer, and advances the commit_lsn/
last_append_lsn watermarks. InternalLogin dispatch and apply
(SDSTOR-22887) and the checkpoint trigger (SDSTOR-22888) are deliberately
left as stubs for follow-up PRs.
- on_commit: validates header/key blob sizes, parses CraftEntryType and
the SyncRSCommitLSNPayload fixed prefix + empty_slots, and detaches
apply_sync_rs_commit_lsn as fire-and-forget (on_commit is a
synchronous HomeStore callback; apply needs to co_await peer fetch +
journal writes). Logs and no-ops on an unrecognized entry type.
- apply_sync_rs_commit_lsn: a client_token mismatch gates the entire
apply (no reconciliation, no catch-up, no watermark advance).
Otherwise, empty_slots are reconciled into empty_lsns_/missing_lsns_,
the newly-spanned range is marked missing, and catch-up via
CraftPeerFetcher::fetch_from_peer + CraftJournalBackend::write_slot is
best-effort: a failed fetch, a failed write, or no peer_fetcher_ wired
at all just leaves the affected LSNs in missing_lsns_ for a later
attempt. commit_lsn/last_append_lsn advance unconditionally afterward
(never decrement), mirroring truncate()'s existing invariant.
- Add volume_error::WRONG_TOKEN for the client_token-mismatch case.
- Add a _PRERELEASE-only test_listener() accessor so tests can drive
on_commit directly.
- New test_craft_raft_entries.cpp (with a MockCraftPeerFetcher) covering
the token gate, empty_slots reconciliation, watermark advance
(including never-decrements), best-effort catch-up (success, fetch
failure, write failure, unwired fetcher), and on_commit dispatch
including malformed-entry rejection.
- guard CraftRaftEntriesTest friend decl with #ifdef _PRERELEASE
- rename OnCommitLogsUnrecognizedEntryType -> OnCommitIgnoresUnrecognizedEntryType
- add tests: mismatched empty_slots count via on_commit, empty_slots
overlapping the same apply's new gap range
- reject the whole apply (new volume_error::INVALID_ENTRY) if
empty_slots has a negative LSN or one above rs_commit_lsn
- validate a peer's fetch_data response against what was requested;
discard the whole batch on an unrequested/duplicate lsn
- document the known use-after-free gap in the detached
apply_sync_rs_commit_lsn coroutine (not fixed yet)
- add tests for both validations
- implement apply_internal_login: overwrite client_token, max-guard
term against regression; synchronous, called directly from
on_commit (no detail::detach -- no I/O to await)
- wire on_commit's InternalLogin dispatch with an exact-size key
check (no variable trailing data, unlike SyncRSCommitLSN)
- fix write()'s pre-existing unlocked read of state_.term -- latent
until now since nothing mutated it; this ticket arms the race
- add client_token()/term() observability accessors
- add tests: dispatch success/wrong-size, second-login replaces
session, term-never-regresses vs token-always-overwrites, write()
term-fencing end-to-end, and cross-entry-type interaction with
apply_sync_rs_commit_lsn's token check
…sns_ - get_rs_commit_lsn() already covered the same snapshot; empty_lsns_ doesn't need ordering.
…timeout CraftPeerFetcher::fetch_from_peer() had no deadline, so an unresponsive peer could hang apply_sync_rs_commit_lsn's catch-up path forever. Adds peer_fetch_timeout_ms (home_blks_config.fbs, default 5000ms) as a CraftReplDev member with a setter, threaded through to fetch_from_peer's new timeout_ms parameter -- kept off the global config singleton so the standalone craft test binaries (which don't link homeblocks_core) still build. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…d_ptr ownership - CraftReplDev now extends std::enable_shared_from_this; apply_sync_rs_commit_lsn opens with `auto self = shared_from_this()` so the detached coroutine holds a strong reference across every co_await, keeping CraftReplDev alive even if the last external owner (e.g. a volume-removal path) drops its shared_ptr mid-apply. Closes the KNOWN GAP flagged in review (PR #2, discussion r3761568811). - CraftReplDev's constructor is now private; construction only via the new CraftReplDev::create() factory, so shared_from_this()'s "must already be shared_ptr-owned" precondition is enforced by the compiler instead of a comment. - Update the four craft test fixtures from make_unique/unique_ptr to CraftReplDev::create()/shared_ptr.
sbinmalek
changed the title
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev/v6.x #176 +/- ##
===========================================
Coverage ? 47.84%
===========================================
Files ? 19
Lines ? 1093
Branches ? 470
===========================================
Hits ? 523
Misses ? 259
Partials ? 311 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the apply side of both CRAFT S5 RAFT entries on top of the S5 infrastructure merged in #172:
on_commitdispatches toapply_sync_rs_commit_lsn, which gatesthe whole apply on a
client_tokenmatch, range-validatesempty_slotsagainstrs_commit_lsn,reconciles them into
empty_lsns_/missing_lsns_, and catches up any local gap viaCraftPeerFetcher::fetch_data. Catch-up is best-effort: a failed fetch, a malformed/unrequested-LSNresponse, a failed
write_slot, or no fetcher wired at all just leaves the LSN inmissing_lsns_for alater attempt.
commit_lsn/last_append_lsnalways advance afterward, never regress.apply_internal_loginoverwritesclient_tokenunconditionally(opaque id, no ordering) and advances
termwith a max-guard against regression, enforcingsingle-writer exclusivity through the existing term-fence check in
write().peer_fetch_timeout_ms, default 5000ms,home_blks_config.fbs) so an unresponsive peer can't hangapply_sync_rs_commit_lsnforever.on_commitcoroutine:CraftReplDevnow derives fromstd::enable_shared_from_this, is constructed only via a newCraftReplDev::create()factory (privateconstructor), and
apply_sync_rs_commit_lsncapturesshared_from_this()so the object stays aliveacross every
co_awaiteven if the last external owner drops itsshared_ptrmid-apply.get_lsns()alias (get_rs_commit_lsn()already covered the identical snapshot)and switches
empty_lsns_tounordered_set(no ordering requirement).Known gaps (tracked, not fixed here)
on_commitstill detaches the apply coroutine fire-and-forget, so a later-committed entry can dispatchbefore this one's effects are fully applied — breaks strict RAFT apply ordering across entries. Real fix
needs a per-device serialized apply queue (flagged as a FIXME in
on_commit).CraftPeerFetcheris still unwired to a real transport — production peer catch-up stays stubbed untilCraftConnector (S9).
Test plan
test_craft_raft_entries.cpp(new): client_token gate, empty_slots range validation +reconciliation, watermark advance (incl. never-decrements), best-effort catch-up (success, fetch
failure, configured timeout threading, malformed/duplicate peer response, write_slot failure, no
fetcher wired), on_commit dispatch for both entry types incl. malformed-entry rejection,
InternalLogin session replacement + term monotonicity + write() term-fencing end-to-end.
test_craft_peer_exchange.cpp/test_craft_truncate.cpp/test_craft_write.cppupdated for theCraftReplDev::create()factory; otherwise unchanged.conan create .full build + ctest.