Skip to content

Commit f1efc2c

Browse files
Michael Norrismeta-codesync[bot]
authored andcommitted
Validate codes/ntotal consistency in Ix2L and IxLa deserialization (#5439)
Summary: Pull Request resolved: #5439 Two faiss deserialization branches accept an index whose `ntotal` is inconsistent with its stored `codes`, so the first `reconstruct()` reads through a null (or out-of-bounds) `codes.data()`: - `Ix2L` (Index2Layer): reads `ntotal` (header), `code_size_1`, `code_size_2`, `code_size`, and `codes` independently. Unlike every sibling `IndexFlatCodes` reader it never checked `codes.size() == ntotal * code_size`. It also never checked that the coarse-code width `code_size_1` matches `q1.nlist` -- and because a PQ with `nbits == 0` yields `code_size_2 == 0`, a forged `code_size_1 == code_size == 0` made the size check vacuous (`0 == ntotal * 0`) while `Level1Quantizer::decode_listno` still reads `coarse_code_size(nlist)` bytes from an empty `codes` buffer (near-null read at IndexIVF.cpp:148). - `IxLa` (IndexLattice): the writer persists the header (including `ntotal`) but never writes the `codes` vector, so a well-formed IxLa index is always empty. A forged `ntotal > 0` leaves `codes` empty and `IndexLattice::sa_decode` hands `codes.data() == nullptr` to `BitstringReader::read`. Adds to Ix2L both the standard `mul_no_overflow` codes-size check and a `code_size_1 == q1.coarse_code_size()` check (so the coarse width matches nlist and the size check can no longer be made vacuous). For IxLa enforces `ntotal == 0` plus header/encoded `d` consistency. All convert a SIGSEGV into a clean FaissException, matching the existing hardening on this surface. Found by agentic fuzzing. Reviewed By: trang-nm-nguyen Differential Revision: D112368337 fbshipit-source-id: 883256965daff9302c505d80e0ac42cf730747ef
1 parent a238933 commit f1efc2c

1 file changed

Lines changed: 20 additions & 0 deletions

File tree

faiss/impl/index_read.cpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2143,6 +2143,16 @@ std::unique_ptr<Index> read_index_up(IOReader* f, int io_flags) {
21432143
r2);
21442144
}
21452145
read_index_header(*idxl, f);
2146+
FAISS_THROW_IF_NOT_FMT(
2147+
idxl->ntotal == 0,
2148+
"IndexLattice deserialization carries no code storage; "
2149+
"ntotal=%zd != 0 is corrupt",
2150+
(size_t)idxl->ntotal);
2151+
FAISS_THROW_IF_NOT_FMT(
2152+
idxl->d == d,
2153+
"IndexLattice header d=%d inconsistent with encoded d=%d",
2154+
idxl->d,
2155+
d);
21462156
READVECTOR(idxl->trained);
21472157
idx = std::move(idxl);
21482158
} else if (h == fourcc("IvSQ")) { // legacy
@@ -2378,6 +2388,10 @@ std::unique_ptr<Index> read_index_up(IOReader* f, int io_flags) {
23782388
READ1(idxp->code_size_1);
23792389
READ1(idxp->code_size_2);
23802390
READ1(idxp->code_size);
2391+
validate_code_size_match(
2392+
idxp->code_size_1,
2393+
idxp->q1.coarse_code_size(),
2394+
"Index2Layer code_size_1");
23812395
validate_code_size_match(
23822396
idxp->code_size_2,
23832397
idxp->pq.code_size,
@@ -2387,6 +2401,12 @@ std::unique_ptr<Index> read_index_up(IOReader* f, int io_flags) {
23872401
idxp->code_size_1 + idxp->code_size_2,
23882402
"Index2Layer");
23892403
read_vector(idxp->codes, f);
2404+
FAISS_THROW_IF_NOT(
2405+
idxp->codes.size() ==
2406+
mul_no_overflow(
2407+
(size_t)idxp->ntotal,
2408+
idxp->code_size,
2409+
"Index2Layer codes"));
23902410
idx = std::move(idxp);
23912411
} else if (
23922412
h == fourcc("IHNf") || h == fourcc("IHNp") || h == fourcc("IHNs") ||

0 commit comments

Comments
 (0)