Pre-flight Checklist
Primary Contact Name
Kelly McKeighan
Primary Contact Email
kelly.l.mckeighan@intel.com
Company / Organization
Intel
Role
Software Engineer
Additional Authors / Contributors
No response
Component
Server
Target Release (Optional)
No response
Roadmap Section
AI & Cloud
Related Issues / Pull Requests / References (Optional)
No response
Executive Summary
Executive Summary
MySQL 9.7 introduced a VECTOR data type, but it remains write-only metadata with no indexing or similarity search. Issue #3 (Native VECTOR Store and VECTOR INDEX Support, Mayank Prasad and Inaam Rana) already proposes closing this gap, and the design has progressed well beyond the initial abstract: Google (Inaam Rana, Julia Offerman) has published detailed InnoDB storage-layer and SQL-layer LLDs for a tree-based ANN index built on ScaNN's k-means clustering (CREATE VECTOR INDEX ... USING TREE), with a PoC design doc at mysql-server#701 and a syntax proposal at mysql-server#702. Notably, the storage layer is built around an abstract VectorIndex interface specifically so that other algorithms, HNSW is mentioned by name in the design discussion, can be added later without touching the SQL layer.
This proposal suggests evaluating Vamana, the graph-based ANN index at the core of Intel's open source Scalable Vector Search (SVS) library, as a candidate second implementation of that VectorIndex interface, complementary to the tree/ScaNN approach rather than competing with it. Vamana already competes well against HNSW on its own merits, and several major systems (Redis, Faiss shipped; OpenSearch, PostgreSQL, Valkey in progress) have adopted it as a pluggable index layer without forking their existing engines. The closest architectural analog is the SVS Extension for PostgreSQL, which layers Vamana on top of pgvector's existing types and distance metrics; benchmarks against pgvector's HNSW show consistent QPS and latency gains, with an optional compression layer (LeanVec/LVQ) providing further improvement while maintaining recall, and a safe fallback to scalar quantization on non-Intel hardware.
The proposed solution is not a competing vector search initiative, but a collaboration with the Issue #3 effort and the broader community on whether Vamana should be one of the algorithms plugged into MySQL's native VECTOR INDEX work, using the PostgreSQL integration as a reference pattern rather than a prescriptive design. Intel would welcome participating as an early contributor now that the VectorIndex interface is taking shape.
The Vector Search Gap
Vector search has become an increasingly common capability among modern databases. The MySQL community is already moving in this direction: Issue #3 proposes evolving MySQL's write-only VECTOR type into a full ACID-compliant vector store with VECTOR INDEX support, distance functions, and ANN search. The design in progress centers on a tree-based index using ScaNN's k-means clustering, deliberately built behind an abstract VectorIndex interface so additional algorithms can be added later. A performant ANN index is genuinely hard to build well, especially competing with engines like PostgreSQL/pgvector that have already invested years into theirs. Rather than build a second implementation from scratch, there's an opportunity to bring in an ANN algorithm that's already proven itself elsewhere as a candidate for that interface.
SVS Open-Source Core to Fill the Gap
Scalable Vector Search (SVS) is built around Vamana, a free, open-source graph-based ANN index. On its own, without any compression, Vamana holds its own against HNSW: in our PostgreSQL benchmark (detailed below) it ran slightly ahead on a 768-dimension dataset and slightly behind on a 1536-dimension dataset. Redis's own blog found a similar result independently. The baseline holds up before any proprietary technology enters the picture.
As a library rather than a standalone service, SVS is designed to embed directly into another system's own index layer. Several major systems have already done exactly that:
Of these, the PostgreSQL extension is worth a closer look: it demonstrates a working integration pattern for Vamana on top of an existing relational engine.
Worked Example: SVS as a PostgreSQL Extension
The SVS Extension for Postgresql layers on top of pgvector, using the same data types and distance metrics, and adds Vamana as a new pluggable index type without forking pgvector itself. Proof of concept is complete and productization is in progress: ACID compliance, crash recovery, and incremental insert/delete are already done, with more work ahead before production-grade.
We benchmarked Vamana against pgvector's HNSW on the Cohere 1M dataset (768 dimensions), both tuned to ~95% recall, using a forked version of VectorDBBench.
| Concurrency |
HNSW QPS |
HNSW p99 (ms) |
SVS Vamana QPS |
SVS Vamana p99 (ms) |
QPS vs. HNSW |
Latency improvement |
| 5 |
1,100 |
6.2 |
1,493 |
4.0 |
1.36x |
35% |
| 20 |
3,319 |
9.7 |
3,564 |
7.4 |
1.07x |
24% |
| 50 |
3,328 |
35.9 |
3,988 |
18.8 |
1.20x |
48% |
Full results, more datasets, and methodology are on the SVS Extension for PostgreSQL wiki page.
This demonstrates an integration pattern, a library plugging into an engine's own extension or index layer. It's not a claim that this exact model drops into MySQL as-is. MySQL doesn't yet have a general-purpose extension framework the way PostgreSQL does. Building one is itself a direction the community has been exploring, and that's exactly where we'd like to collaborate, on what a MySQL-side interface for this kind of integration should look like.
What Intel Is Proposing
We'd like to explore whether there's community interest in bringing Vamana, and the SVS work behind it, into MySQL's native vector search effort.
The architecture decision between a custom InnoDB page format and a hidden sub_table (mysql-server#710) is still open. We'd build Vamana against whichever direction the community settles on.
What we could offer: an adapter for Vamana's non-local mutation pattern (a single insert can touch several unrelated existing nodes' neighbor lists, unlike IVF's one-row-one-partition locality), its MVCC visibility during graph traversal, and its memory footprint, a working answer to Jakub Łopuszański's question on mysql-server#701: "is anyone willing to contribute HNSW to give us some idea of what's required?" We'd build this as an actual PoC against the Vector_index_interface.
One thing would help us align that work with what's already in flight on Issue #3: whether Google's IndexRegistry-style memory accounting for their tree centroids is meant to be a reusable pattern. If so, we'd build Vamana's graph memory accounting against it rather than a separate scheme.
User / Developer Stories
No response
Proposed Scope
No response
Out of Scope / Future Work
No response
References
No response
Functional Requirements
No response
Non-functional Requirements
No response
Impact Areas
Summary of the Approach
No response
User Interface
No response
Configuration / Knobs
No response
Observability
No response
User Procedure
No response
Security Considerations
No response
Compatibility and Behavior Changes
No response
Block Diagram
No response
Interface Specification
No response
Proposed Implementation Plan
No response
QA Notes
No response
Pre-flight Checklist
Primary Contact Name
Kelly McKeighan
Primary Contact Email
kelly.l.mckeighan@intel.com
Company / Organization
Intel
Role
Software Engineer
Additional Authors / Contributors
No response
Component
Server
Target Release (Optional)
No response
Roadmap Section
AI & Cloud
Related Issues / Pull Requests / References (Optional)
No response
Executive Summary
Executive Summary
MySQL 9.7 introduced a VECTOR data type, but it remains write-only metadata with no indexing or similarity search. Issue #3 (Native VECTOR Store and VECTOR INDEX Support, Mayank Prasad and Inaam Rana) already proposes closing this gap, and the design has progressed well beyond the initial abstract: Google (Inaam Rana, Julia Offerman) has published detailed InnoDB storage-layer and SQL-layer LLDs for a tree-based ANN index built on ScaNN's k-means clustering (
CREATE VECTOR INDEX ... USING TREE), with a PoC design doc at mysql-server#701 and a syntax proposal at mysql-server#702. Notably, the storage layer is built around an abstractVectorIndexinterface specifically so that other algorithms, HNSW is mentioned by name in the design discussion, can be added later without touching the SQL layer.This proposal suggests evaluating Vamana, the graph-based ANN index at the core of Intel's open source Scalable Vector Search (SVS) library, as a candidate second implementation of that
VectorIndexinterface, complementary to the tree/ScaNN approach rather than competing with it. Vamana already competes well against HNSW on its own merits, and several major systems (Redis, Faiss shipped; OpenSearch, PostgreSQL, Valkey in progress) have adopted it as a pluggable index layer without forking their existing engines. The closest architectural analog is the SVS Extension for PostgreSQL, which layers Vamana on top of pgvector's existing types and distance metrics; benchmarks against pgvector's HNSW show consistent QPS and latency gains, with an optional compression layer (LeanVec/LVQ) providing further improvement while maintaining recall, and a safe fallback to scalar quantization on non-Intel hardware.The proposed solution is not a competing vector search initiative, but a collaboration with the Issue #3 effort and the broader community on whether Vamana should be one of the algorithms plugged into MySQL's native VECTOR INDEX work, using the PostgreSQL integration as a reference pattern rather than a prescriptive design. Intel would welcome participating as an early contributor now that the
VectorIndexinterface is taking shape.The Vector Search Gap
Vector search has become an increasingly common capability among modern databases. The MySQL community is already moving in this direction: Issue #3 proposes evolving MySQL's write-only VECTOR type into a full ACID-compliant vector store with VECTOR INDEX support, distance functions, and ANN search. The design in progress centers on a tree-based index using ScaNN's k-means clustering, deliberately built behind an abstract
VectorIndexinterface so additional algorithms can be added later. A performant ANN index is genuinely hard to build well, especially competing with engines like PostgreSQL/pgvector that have already invested years into theirs. Rather than build a second implementation from scratch, there's an opportunity to bring in an ANN algorithm that's already proven itself elsewhere as a candidate for that interface.SVS Open-Source Core to Fill the Gap
Scalable Vector Search (SVS) is built around Vamana, a free, open-source graph-based ANN index. On its own, without any compression, Vamana holds its own against HNSW: in our PostgreSQL benchmark (detailed below) it ran slightly ahead on a 768-dimension dataset and slightly behind on a 1536-dimension dataset. Redis's own blog found a similar result independently. The baseline holds up before any proprietary technology enters the picture.
As a library rather than a standalone service, SVS is designed to embed directly into another system's own index layer. Several major systems have already done exactly that:
Of these, the PostgreSQL extension is worth a closer look: it demonstrates a working integration pattern for Vamana on top of an existing relational engine.
Worked Example: SVS as a PostgreSQL Extension
The
SVS Extension for Postgresqllayers on top of pgvector, using the same data types and distance metrics, and adds Vamana as a new pluggable index type without forking pgvector itself. Proof of concept is complete and productization is in progress: ACID compliance, crash recovery, and incremental insert/delete are already done, with more work ahead before production-grade.We benchmarked Vamana against pgvector's HNSW on the Cohere 1M dataset (768 dimensions), both tuned to ~95% recall, using a forked version of VectorDBBench.
Full results, more datasets, and methodology are on the SVS Extension for PostgreSQL wiki page.
This demonstrates an integration pattern, a library plugging into an engine's own extension or index layer. It's not a claim that this exact model drops into MySQL as-is. MySQL doesn't yet have a general-purpose extension framework the way PostgreSQL does. Building one is itself a direction the community has been exploring, and that's exactly where we'd like to collaborate, on what a MySQL-side interface for this kind of integration should look like.
What Intel Is Proposing
We'd like to explore whether there's community interest in bringing Vamana, and the SVS work behind it, into MySQL's native vector search effort.
The architecture decision between a custom InnoDB page format and a hidden
sub_table(mysql-server#710) is still open. We'd build Vamana against whichever direction the community settles on.What we could offer: an adapter for Vamana's non-local mutation pattern (a single insert can touch several unrelated existing nodes' neighbor lists, unlike IVF's one-row-one-partition locality), its MVCC visibility during graph traversal, and its memory footprint, a working answer to Jakub Łopuszański's question on mysql-server#701: "is anyone willing to contribute HNSW to give us some idea of what's required?" We'd build this as an actual PoC against the
Vector_index_interface.One thing would help us align that work with what's already in flight on Issue #3: whether Google's
IndexRegistry-style memory accounting for their tree centroids is meant to be a reusable pattern. If so, we'd build Vamana's graph memory accounting against it rather than a separate scheme.User / Developer Stories
No response
Proposed Scope
No response
Out of Scope / Future Work
No response
References
No response
Functional Requirements
No response
Non-functional Requirements
No response
Impact Areas
Summary of the Approach
No response
User Interface
No response
Configuration / Knobs
No response
Observability
No response
User Procedure
No response
Security Considerations
No response
Compatibility and Behavior Changes
No response
Block Diagram
No response
Interface Specification
No response
Proposed Implementation Plan
No response
QA Notes
No response