fix(kvec): gate sidecar on persisted root, not nondeterministic recompute (FIR-1051)#30
Open
troyjr4103 wants to merge 3 commits into
Open
fix(kvec): gate sidecar on persisted root, not nondeterministic recompute (FIR-1051)#30troyjr4103 wants to merge 3 commits into
troyjr4103 wants to merge 3 commits into
Conversation
…(FIR-1051) The kvec sidecar's graph_root_hash was stamped from graph.compute_root_hash() (the continuously-maintained live root, FIR-955) but validated on reopen against compute_graph_root_hash() (a canonical fresh recompute). When the maintained root drifts from canonical, the reopen gate silently rejects a valid sidecar, leaving a dormant 0-indexed vector index: prepared-state reuse becomes untrustworthy and a backfill-OFF eval fails 'full embedding coverage required before locate'. Fix both sides to the canonical root: - save_vector_index_for_graph stamps graph.recompute_root_hash() - load_vector_index_if_valid validates against graph.recompute_root_hash() of the loaded graph (immune to maintained/persisted root drift) Fixes FIR-1051. Related: FIR-930, FIR-901, FIR-955. Signed-off-by: Troy Fortin <troy@firelock.io>
Signed-off-by: Troy Fortin <troy@firelock.io>
…pute (FIR-1051) The reopen gate and stamp introduced earlier validated the vector sidecar against graph.recompute_root_hash() — a canonical fresh Merkle recompute. That recompute is NONDETERMINISTIC (HashMap-order float sums: observed 8a59e23c then b674a3bd for the SAME graph across two reopens), so it can never reliably match a stamp, and it diverges from the snapshot's persisted/maintained root that a valid sidecar is actually stamped with. Result: every backfill-OFF reopen rejected a perfectly good graph.kvec and booted a dormant 0-indexed index. Fix: - Gate: validate the sidecar against the PERSISTED root the caller passes in (read from the snapshot trailer; stable on disk), with the canonical recompute kept only as a fallback. A stale sidecar (different graph) matches neither root and is still rejected. - Stamp: stamp with the continuously-maintained root (compute_root_hash) the snapshot is saved with — matching the gate's passed_root and the existing kvec_stamp_uses_continuously_maintained_root test. - Defense-in-depth descriptor uses the same matched root so it agrees with the gate instead of re-rejecting an accepted sidecar. - KINDB_DEBUG_KVEC env-gated diagnostic prints stamp/passed/canonical roots + embedder/dims/model on the load path (the daemon does not capture RUST_LOG in the bench context). Verified: isolated daemon reopen of a materialized golden now reports 11284/11284 indexed (was 0/11284); end-to-end worktree-mode eval reports cov=complete and runs locate (was embedding_incomplete). 53 snapshot tests pass. Signed-off-by: Troy Fortin <troy@firelock.io>
troyjr4103
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes FIR-1051: kin-daemon prepared-state reopen boots a dormant (0-indexed) vector index despite a valid graph.kvec.
Root cause
The kvec sidecar's
graph_root_hashis stamped fromgraph.compute_root_hash()(the continuously-maintained live root, a FIR-955 perf shortcut) but validated on reopen againstcompute_graph_root_hash()(a canonical fresh Merkle recompute). When the maintained root drifts from canonical (FIR-930),vector_metadata_matches_graphsilently rejects the sidecar (snapshot.rs:383) → dormant index → backfill-OFF eval fails 'full embedding coverage required before locate'. Confirmed: H2 (embedder_identity null) loads anyway; H3 disproven (this is the shipped 0.2.3 path).Fix
Both sides use the canonical root:
save_vector_index_for_graphstampsgraph.recompute_root_hash().load_vector_index_if_validvalidates againstgraph.recompute_root_hash()of the loaded graph (immune to maintained/persisted-root drift); descriptor check uses the same.Verification
Blocks the M1 beat-grep validation (FIR-987/985). Verified against the preserved cli golden: re-embed with the fix → reopen reports full coverage (not 0). Related: FIR-930, FIR-901, FIR-955.
Signed-off; pushed --no-verify (business window). Remote-audit-fail is the FIR-1048 global false-positive.