QVAC-20631 TurboVec: fabric vector index (CPU)#185
Draft
dev-nid wants to merge 15 commits into
Draft
Conversation
Add a standalone ggml vector index API and flat f32 TurboVec POC implementation for fabric embedding lookup. Includes .tvim v1 persistence, duplicate-safe atomic adds, scalar top-k search, and focused test coverage.
Adds the fabric vector index API with f32/q8 storage, NEON-accelerated q8 search, crash-safe checksummed .tvim persistence, and regression coverage for corruption and migration paths. Assisted-by: GPT-5.5
Adds isolated fault-injection coverage for vector index OOM and interrupted-write paths, including partial batch rollback, failed snapshot preservation, and temp-file cleanup. Assisted-by: GPT-5.5
Adds runtime-dispatched AVX2 acceleration for q8 vector index search on x86 while preserving scalar fallback and the existing ARM NEON path. Assisted-by: GPT-5.5
Adds quantized q8/q4 storage, crash-safe persistence with checksums, filtered and prepared-filtered search, delta logs with compaction, IVF-flat ANN search, SIMD q8/q4 kernels, and thread-safe concurrent reads. Assisted-by: GPT-5.5
Adds q8/q4 storage and search, filtered search, IVF-flat ANN, incremental persistence, mmap loading, tombstone deletes, and in-memory compaction. Covers crash-safe snapshots, checksums, delta replay/compaction, SIMD kernels, fault tests, and vector index benchmarks. Assisted-by: GPT-5.5
Adds mmap load timing, delete-heavy compaction metrics, multi-distribution q4/q8 quality checks, and benchmark-only q4 calibration experiments. Assisted-by: GPT-5.5
Move the vector index implementation out of ggml-base into its own exported target, and tighten API invariants around prepared filters, reserved sentinel IDs, and delta-log compaction recovery.
Fix delta-log consistency issues around committed mutation failures, shared log serialization, deterministic quantized replay, and unsafe compaction paths. Also tighten prepared filter validation and CMake package/linkage behavior.
Return success when logged mutations or snapshot replacements have already committed despite late durability checks, and cache validated delta tails to avoid rescanning the full log on repeated appends.
Tighten mmap snapshot safety, delta durability/error handling, IVF probing, length limits, and package exports while expanding vector-index tests and benchmark coverage for edge cases, quantized persistence, and filtered q4 paths.
Keep logged mutation completion checks under the delta log lock, and clarify vector-index API docs around SIMD dispatch, scoring, IVF probes, and mmap endian requirements.
Clarify delta-log writer, replay, and exact-search contracts in the public vector-index API. Add AVX2 acceleration for q4 dot-product search and report the q4 kernel in the vector-index benchmark.
Fix Turbovec delta-log correctness issues by removing stale tail assumptions, adding v2 state tokens, serializing load-with-delta against compaction, making replayed removes strict, and reporting partial compaction explicitly. Also split AVX2 kernels for runtime dispatch, add pkg-config metadata, improve Windows directory durability, and expand regression/fault coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TurboVec: fabric vector index (CPU)
Adds a standalone
ggml-vector-indexC API and a production-ready CPU implementation for fabric's embedding lookup / retrieval layer. This branch is the CPU-only counterpart ofturbovec(which adds Metal on top of the same base).Summary
Introduces a new
ggml_vec_index_*C API (ggml/include/ggml-vector-index.h) and its implementation (ggml/src/ggml-vector-index.cpp), plus full unit, fault, and benchmark coverage..tvimv1 snapshots with checksums, delta replay, and compaction on write. Fixed little-endian on disk. Supportsmmaploading.Test plan
test-vector-index— API correctness across f32 / q8 / q4, persistence round-trips, migration, corruption, filtered / prepared-filtered search, IVF-flat.test-vector-index-faults— OOM and interrupted-write recovery paths.bench-vector-index— flat search, filtered search, IVF, mmap load, and delete-heavy compaction.turbovecbase commit (c947e5ca2).Notes
turbovec(5d59a1bMetal top-k,a94f463Metal support files) so it can ship as a CPU-only rollout..tvimv1, fixed little-endian.Related PR
embed: tetherto/qvac#3292