Fix flaky HNSW vector index unit test - #808
Merged
Merged
Conversation
Two independent fragilities surfaced intermittently in HierarchicalNavigableSmallWorld indexing > can delete and update and search with vector index with one dimension (HNSW uses Math.random() for level assignment, so the failure mode varies across runs): - verifySearch asserted exact vector equality between HNSW top-5 and brute-force top-5. HNSW is approximate; under heavy delete/update churn with many near-duplicate vectors, recall@5 can drop below 100%. Replaced with a per-rank distance tolerance check (HNSW distance must be within 0.05 of brute-force best at that rank). - validateConnectivity crashed (TypeError reading 'level' on undefined) when BFS traversed stale neighbor IDs left over from deletes — HNSW only scrubs the deleted node's own connection list, and asymmetric edges can survive. It also inflated visited.size by recording stale neighbor IDs before confirming the underlying node existed, breaking isFullyConnected. BFS now uses safeGetSync, skips missing nodes, and tracks queue membership in a separate set so visited reflects real nodes only. Verified stable: 50 consecutive runs pass; all 536 resources tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Reviewed; no blockers found. |
Pre-existing Prettier drift on main from 46109e6 was breaking the Format Check job on every new PR. Pure whitespace reformat, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kriszyp
marked this pull request as ready for review
May 27, 2026 00:55
cb1kenobi
approved these changes
May 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the flaky
HierarchicalNavigableSmallWorld indexing > can delete and update and search with vector index with one dimensiontest seen on unrelated PRs (e.g. run 26469662007).Purpose
Two independent fragilities surfaced intermittently. HNSW level assignment uses
Math.random(), so each run produces a different graph topology and the failure mode varied across runs (~15% local repro rate):verifySearchusedassert.deepEqualon HNSW top‑5 vs brute‑force top‑5. HNSW is approximate; under the test's stress sequence (200 inserts → delete 100 → upsert all 200 with many near‑duplicate modular‑arithmetic vectors), recall@5 can dip below 100%. Replaced with a per‑rank distance tolerance (HNSW distance must be within0.05of brute‑force best at that rank).validateConnectivitycrashed on stale neighbor refs. HNSW deletes only scrub the deleted node's own connection list, and asymmetric edges can survive (this is logged asasymmetry detectedwarnings). When BFS traversed a stale neighbor ID,getSyncreturnedundefinedand the BFS threwTypeError: Cannot read properties of undefined (reading 'level'). It also inflatedvisited.sizeby adding stale IDs before confirming the node existed, which brokeisFullyConnected. BFS now usessafeGetSync, skips missing nodes, and tracks queue membership separately sovisitedreflects real nodes only.Where to focus review
0.05) in unitTests/resources/vectorIndex.test.js: chose this empirically — comfortably covers observed approximation gaps (largest delta seen was ~0.016 cosine) without masking real regressions. Open to a tighter value if reviewers prefer.test:integration:allsince no integration tests exercise HNSW (grepconfirms zeroHNSW/vectorIndexrefs underintegrationTests/).test:unit:resources(536 tests) all pass.Verification
assert(results.length >= 5)bounds check).🤖 Generated with Claude Opus 4.7