Add memory-mapped read path - #8
Merged
Merged
Conversation
Replace the per-element ReadSingle/Write(float) loops in HnswIndex.Save and Load with bulk MemoryMarshal byte-span reads and writes. On a 20,487-vector, 64-dim index this cuts Load from ~59ms to ~22ms (min, ~2-3x) by avoiding millions of scalar BinaryReader/BinaryWriter calls and their per-call bounds checks. The on-disk bytes are unchanged on the little-endian platforms .NET targets (BinaryWriter.Write(float) and MemoryMarshal both produce little-endian), so the format is byte-identical and existing v3 indexes load correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Introduce SlotVector(slot) and SlotLinks(slot, layer) and read every vector and neighbor list in SearchGreedy, SearchLayer, SelectNeighbors and Search through them. This centralizes per-slot storage access so the backing store can later be swapped for a memory-mapped level-0 block without touching the search algorithms. Behavior is unchanged; the parity test suite is green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the single float[] backing the normalized search vectors with a chunked, slot-addressed VectorBlock. Each chunk stays well under the .NET array length limit so the index scales to very large repos, and every slot exposes a contiguous span ready for a future memory-mapped backing. The v3 on-disk format and all read/write semantics are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop Node.OriginalVector so each node keeps only the stored (search) vector, halving vector memory and producing the contiguous single-vector-per-slot layout needed for a memory-mapped backing. Bump the on-disk format to v4 which writes one vector per node; v1-v3 files still load, with the older duplicate pre-normalization copy read and discarded. For DistanceMetric.Cosine this changes ExportItems/TryGetVector to return the unit-normalized stored vector instead of the original input magnitude, matching hnswlib's getDataByLabel. Re-adding a normalized vector is a no-op normalization so rebuild stays stable. Other metrics are unaffected. Docs and tests updated, plus a regression test that loads a hand-crafted v3 stream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restructure the v4 on-disk format into sections: a contiguous, fixed-stride vector block followed by a separate graph section. This lets LoadMapped memory- map the vector block (the bulk of the data) instead of reading it into the managed heap, while the graph is still loaded into RAM so search stays fast. The vector store is now an abstraction with a chunked heap implementation (build and default Load) and a MemoryMappedFile-backed implementation addressed with long offsets, so the vector section can exceed the .NET array length limit while each per-slot span stays within it. A mapped index is read-only (Add throws) and owns the mapping, so HnswIndex is now IDisposable. Formats v1-v3 still load via the previous interleaved reader. Adds LoadMapped round-trip/read-only/dispose tests (48 total) and README docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
LoadCore reads every record payload off disk and deserializes all N records into objects up front, so a large collection pays its full size in load time and managed allocation before the first query. This adds a path-based load that memory-maps the file instead. - HnswIndex.LoadMapped(path, offset): map the vector section of an index that begins at a byte offset within a larger container file. - HnswCollection.Load(string [, offset], [context]): map the snapshot, read framing eagerly (keys materialized, record payloads located but not read), defer TRecord deserialization to first access from the mapped region, and map the embedded index's vectors. Records stay off the managed heap; the index owns the vectors, so the collection keeps none. The result is read-only. - Entry now supports lazy materialization (thread-safe, cached); the mapping lifetime is owned by HnswCollectionData and released on reload, collection deletion, or store disposal. - Framing corruption is still detected at load; record-payload JSON validity is validated on access for the mapped path (documented). Stream-based Load and its corruption tests are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a memory-mapped (mmap) load path for Hnsw.Net indexes and vector-store snapshots to reduce cold-start load time and managed allocations by keeping vectors and record payloads off-heap and paging them in on demand. It also introduces an updated on-disk index format to support mmap-friendly, contiguous vector storage and to lift large-index size limitations.
Changes:
- Add
HnswIndex.LoadMapped(...)to memory-map the saved vector section while loading the graph into memory, making the mapped index read-only and disposable. - Add mmap-backed
HnswCollection.Load(path, ...)overloads that validate framing/keys eagerly but lazily deserialize record payload JSON on first access, with mapping lifetime owned by the collection/store data. - Update index persistence to format v4 with a contiguous vector section plus graph section, and speed up I/O via bulk float reads/writes; add/adjust tests and documentation accordingly.
Show a summary per file
| File | Description |
|---|---|
| tests/Hnsw.Net.Tests/VectorStoreMmapTests.cs | Adds tests covering mmap snapshot load, lazy record materialization, embedded-at-offset loads, parity vs stream load, and mapping release on dispose. |
| tests/Hnsw.Net.Tests/HnswIndexTests.cs | Adds tests for LoadMapped parity + read-only behavior and updates expectations around exported (stored/normalized) vectors; adds a legacy v3 load test. |
| src/Hnsw.Net/VectorData/HnswVectorStore.cs | Ensures collection deletion and store disposal dispose underlying collection data to release mappings. |
| src/Hnsw.Net/VectorData/HnswCollectionData.cs | Introduces mapping ownership (MappedRecordFile) and lazy record materialization via factory-backed entries; adds disposal. |
| src/Hnsw.Net/VectorData/HnswCollection.cs | Adds mmap-based load overloads (including embedded offset), installs/disposes mappings safely when replacing loaded state, and introduces span-based JSON deserialization helpers. |
| src/Hnsw.Net/HnswIndex.cs | Implements format v4 vector section + graph section, chunked vector storage, mmap vector storage, LoadMapped, and updates search path to read vectors via an abstraction. |
| src/Hnsw.Net/Hnsw.Net.csproj | Enables unsafe blocks to support pointer-based mmap spans. |
| README.md | Updates ExportItems semantics documentation and documents LoadMapped usage and constraints. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 8/8 changed files
- Comments generated: 5
Validate every persisted link index (0 <= neighbor < Count) and bound link counts while loading. Unchecked indices reached MappedVectorBlock, which builds a span from a raw pointer with no bounds check, so a corrupt link could read arbitrary mapped memory or fault the process instead of throwing. Restore the explicit little-endian on-disk vector format that the pre-bulk-IO code guaranteed: bulk byte copy on little-endian hosts (the only platforms .NET supports) with a scalar fallback otherwise. LoadMapped reinterprets mapped bytes in place, so it now rejects big-endian platforms rather than returning silently wrong results. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Make HnswIndex.Dispose idempotent (ReaderWriterLockSlim must not be disposed twice). - Enforce the read-only contract in MarkDeleted/UnmarkDeleted, not just Add. - Validate the vector section in LoadMapped: reject negative count/dimension, guard the size multiplication against overflow, and ensure it fits the file. - Dispose HnswCollectionData under its Lock and clear fields so teardown cannot race in-flight operations using unsafe mapped spans. - Use 'using' for stores in mmap tests so disposal cannot mask assertion failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Reject a non-positive dimension and negative count in ReadHeader so both Load and LoadMapped fail with InvalidDataException instead of DivideByZeroException or a silently-empty index on corrupt input. - Dispose the partially-constructed index if creating the memory mapping fails. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Memory-mapped read path
Summary
Adds a memory-mapped load path so consumers can open large snapshots without reading the whole file into managed memory, plus the supporting format/perf work for the 0.1.3 release. Cold-start load cost becomes largely independent of index size: vectors and record payloads fault in on demand instead of being eagerly deserialized to the heap.
What's in this PR
HnswIndex.LoadMapped— opens an index by path, maps the contiguous vector section read-only (addressed by absolute offset), and reads the graph after. An offset overload (LoadMapped(path, baseOffset)) supports indexes embedded inside a larger file.HnswCollection.Load(path, …)— maps the snapshot, eagerly validates framing (magic/version/dims/counts/lengths/bounds) and keys, but defers each record payload's JSON deserialization to first access. Vectors stay off-heap (ids validated viaindex.Contains, notTryGetVector). Mapping lifetime is owned byHnswCollectionData/HnswVectorStoreand released on dispose/delete.VectorBlock, which also lifts the single-byte[]/2 GB array-size ceiling for very large indexes. Search hot path routed through per-slot accessors.Save/Loadvia bulk float I/O.All new public APIs are additive; PackageValidation passes against the 0.1.2 baseline.
Compatibility
Load(Stream)and all corruption-detection tests are unchanged. The mmap path is an additive overload; mutable/tracking consumers keep the eager load (a mapped index is read-only).Benchmarks
Synthetic runtime-libraries scale (read path is content-independent):
End-to-end via the downstream consumer (13,448 passages, 26 MB composite index): cold load ~310 ms → ~160 ms (1.9×), managed alloc 402 → 61 MiB (6.6×), peak working set 172 → 116 MiB; identical results.
Tests
52/52 green, including new
VectorStoreMmapTests(round-trip + search, by-reflection records, embedded-at-offset, parity with stream load, and mapping-released-on-dispose so the file can be deleted).