Skip to content

Add half-float (FP16) storage support for vectors #15549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Pulkitg64 wants to merge 24 commits into
base: main
Choose a base branch
from
Open

Add half-float (FP16) storage support for vectors #15549

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2854714
Add float16 support
Dec 22, 2025
f0f6224
Do not add support for float16
Dec 22, 2025
09cb1ca
Add support in test cases as well
Dec 22, 2025
e6d7da0
Add support in test cases
Dec 22, 2025
237a506
Fixed byteSize calculation
Dec 22, 2025
232cd17
tidy
Dec 22, 2025
2145c90
Fixed float16 vector generator
Dec 22, 2025
d86281c
Fixed test cases
Dec 22, 2025
076a51c
Tidy
Jan 5, 2026
7e4689f
Add java docs
Jan 5, 2026
6599f80
Try Float16Vector for vectorscore computation
Pulkitg64 Jan 15, 2026
f0a71d0
Fix dotProduct function when memorysegment is input
Pulkitg64 Jan 24, 2026
dcbd9fc
Add bulk vector support for float16 dot-product scoring
Feb 3, 2026
e589f82
tidy
Feb 3, 2026
8a48524
cleanup changes and tidy
Feb 9, 2026
b85137a
More cleanups and added missing support for float16 vector scoring
Feb 9, 2026
c86dfe1
Remove panama and memory segment implementation of float16
Feb 10, 2026
f316095
Merge branch 'main' into float16
Feb 10, 2026
5d2a9a5
Added entry to changes.txt and fixed some writer classes
Feb 10, 2026
a08cba9
Cleanup vectorUtil class and added OffHeapScalarQuantizedFloat16Vecto…
Feb 10, 2026
24b2a6e
Merge branch 'main' into float16
Jun 1, 2026
d415c07
Address review comments from msokolov
Jun 1, 2026
9f236d0
Fix fp16 COSINE quantization: re-normalize inflated vector
Jun 1, 2026
2645e2f
Lucene104ScalarQuantizedVectorsWriter and related refactoring
Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lucene/CHANGES.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,8 @@ New Features
* GITHUB#16110: Add count() to BayesianScoreQuery.BayesianScoreWeight, delegating to
the inner weight. (Prithvi S)

* GITHUB#15549: Add fp16 vector encoding support. (Pulkit Gupta)

Improvements
---------------------
* GITHUB#15948: Improve BayesianScoreQuery and LogOddsFusionQuery with base rate prior,
Expand Down
7 changes: 7 additions & 0 deletions ...rc/java/org/apache/lucene/backward_codecs/lucene102/Lucene102BinaryFlatVectorsScorer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,13 @@ public RandomVectorScorer getRandomVectorScorer(
return nonQuantizedDelegate.getRandomVectorScorer(similarityFunction, vectorValues, target);
}

@Override
public RandomVectorScorer getRandomVectorScorer(
VectorSimilarityFunction similarityFunction, KnnVectorValues vectorValues, short[] target)
throws IOException {
return null;
}

@Override
public String toString() {
return "Lucene102BinaryFlatVectorsScorer(nonQuantizedDelegate=" + nonQuantizedDelegate + ")";
Expand Down
11 changes: 11 additions & 0 deletions ...va/org/apache/lucene/backward_codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import org.apache.lucene.index.DocsWithFieldSet;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.KnnVectorValues;
Expand Down Expand Up @@ -210,6 +211,11 @@ public RandomVectorScorer getRandomVectorScorer(String field, byte[] target) thr
return rawVectorsReader.getRandomVectorScorer(field, target);
}

@Override
public RandomVectorScorer getRandomVectorScorer(String field, short[] target) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void checkIntegrity() throws IOException {
rawVectorsReader.checkIntegrity();
Expand Down Expand Up @@ -252,6 +258,11 @@ public ByteVectorValues getByteVectorValues(String field) throws IOException {
return rawVectorsReader.getByteVectorValues(field);
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, byte[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand Down
12 changes: 12 additions & 0 deletions ...codecs/src/java/org/apache/lucene/backward_codecs/lucene90/Lucene90HnswVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -242,6 +243,11 @@ public ByteVectorValues getByteVectorValues(String field) {
throw new UnsupportedOperationException();
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand Down Expand Up @@ -279,6 +285,12 @@ public void search(String field, byte[] target, KnnCollector knnCollector, Accep
throw new UnsupportedOperationException();
}

@Override
public void search(String field, short[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
throw new UnsupportedOperationException();
}

private OffHeapFloatVectorValues getOffHeapVectorValues(FieldEntry fieldEntry)
throws IOException {
IndexInput bytesSlice =
Expand Down
12 changes: 12 additions & 0 deletions ...codecs/src/java/org/apache/lucene/backward_codecs/lucene91/Lucene91HnswVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -238,6 +239,11 @@ public ByteVectorValues getByteVectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand All @@ -263,6 +269,12 @@ public void search(String field, byte[] target, KnnCollector knnCollector, Accep
throw new UnsupportedOperationException();
}

@Override
public void search(String field, short[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
throw new UnsupportedOperationException();
}

private OffHeapFloatVectorValues getOffHeapVectorValues(FieldEntry fieldEntry)
throws IOException {
IndexInput bytesSlice =
Expand Down
12 changes: 12 additions & 0 deletions ...codecs/src/java/org/apache/lucene/backward_codecs/lucene92/Lucene92HnswVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -235,6 +236,11 @@ public ByteVectorValues getByteVectorValues(String field) {
throw new UnsupportedOperationException();
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand All @@ -260,6 +266,12 @@ public void search(String field, byte[] target, KnnCollector knnCollector, Accep
throw new UnsupportedOperationException();
}

@Override
public void search(String field, short[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
throw new UnsupportedOperationException();
}

private HnswGraph getGraph(FieldEntry entry) throws IOException {
IndexInput bytesSlice =
vectorIndex.slice("graph-data", entry.vectorIndexOffset, entry.vectorIndexLength);
Expand Down
13 changes: 13 additions & 0 deletions ...codecs/src/java/org/apache/lucene/backward_codecs/lucene94/Lucene94HnswVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -177,6 +178,7 @@ private void validateFieldEntry(FieldInfo info, FieldEntry fieldEntry) {
switch (info.getVectorEncoding()) {
case BYTE -> Byte.BYTES;
case FLOAT32 -> Float.BYTES;
case FLOAT16 -> Short.BYTES;
};
long vectorBytes = Math.multiplyExact((long) dimension, byteSize);
long numBytes = Math.multiplyExact(vectorBytes, fieldEntry.size);
Expand Down Expand Up @@ -269,6 +271,11 @@ public ByteVectorValues getByteVectorValues(String field) throws IOException {
return OffHeapByteVectorValues.load(fieldEntry, vectorData);
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand Down Expand Up @@ -307,6 +314,12 @@ public void search(String field, byte[] target, KnnCollector knnCollector, Accep
vectorValues.getAcceptOrds(acceptDocs.bits()));
}

@Override
public void search(String field, short[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
throw new UnsupportedOperationException();
}

private HnswGraph getGraph(FieldEntry entry) throws IOException {
IndexInput bytesSlice =
vectorIndex.slice("graph-data", entry.vectorIndexOffset, entry.vectorIndexLength);
Expand Down
1 change: 1 addition & 0 deletions ...-codecs/src/java/org/apache/lucene/backward_codecs/lucene94/OffHeapFloatVectorValues.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ static OffHeapFloatVectorValues load(
switch (fieldEntry.vectorEncoding()) {
case BYTE -> fieldEntry.dimension();
case FLOAT32 -> fieldEntry.dimension() * Float.BYTES;
case FLOAT16 -> fieldEntry.dimension() * Short.BYTES;
};
if (fieldEntry.docsWithFieldOffset() == -1) {
return new DenseOffHeapVectorValues(
Expand Down
13 changes: 13 additions & 0 deletions ...codecs/src/java/org/apache/lucene/backward_codecs/lucene95/Lucene95HnswVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -177,6 +178,7 @@ private void validateFieldEntry(FieldInfo info, FieldEntry fieldEntry) {
switch (info.getVectorEncoding()) {
case BYTE -> Byte.BYTES;
case FLOAT32 -> Float.BYTES;
case FLOAT16 -> Short.BYTES;
};
long vectorBytes = Math.multiplyExact((long) dimension, byteSize);
long numBytes = Math.multiplyExact(vectorBytes, fieldEntry.size);
Expand Down Expand Up @@ -289,6 +291,11 @@ public ByteVectorValues getByteVectorValues(String field) throws IOException {
vectorData);
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
Expand Down Expand Up @@ -345,6 +352,12 @@ public void search(String field, byte[] target, KnnCollector knnCollector, Accep
vectorValues.getAcceptOrds(acceptDocs.bits()));
}

@Override
public void search(String field, short[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)
throws IOException {
throw new UnsupportedOperationException();
}

/** Get knn graph values; used for testing */
@Override
public HnswGraph getGraph(String field) throws IOException {
Expand Down
11 changes: 11 additions & 0 deletions ...java/org/apache/lucene/backward_codecs/lucene99/Lucene99ScalarQuantizedVectorsReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.FieldInfo;
import org.apache.lucene.index.FieldInfos;
import org.apache.lucene.index.Float16VectorValues;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.IndexFileNames;
import org.apache.lucene.index.SegmentReadState;
Expand Down Expand Up @@ -227,6 +228,11 @@ public ByteVectorValues getByteVectorValues(String field) throws IOException {
return rawVectorsReader.getByteVectorValues(field);
}

@Override
public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
return rawVectorsReader.getFloat16VectorValues(field);

@msokolov msokolov Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to support this in the backwards codec?

}

private static IndexInput openDataInput(
SegmentReadState state,
int versionMeta,
Expand Down Expand Up @@ -290,6 +296,11 @@ public RandomVectorScorer getRandomVectorScorer(String field, float[] target) th
return vectorScorer.getRandomVectorScorer(fieldEntry.similarityFunction, vectorValues, target);
}

@Override
public RandomVectorScorer getRandomVectorScorer(String field, short[] target) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public RandomVectorScorer getRandomVectorScorer(String field, byte[] target) throws IOException {
return rawVectorsReader.getRandomVectorScorer(field, target);
Expand Down
6 changes: 6 additions & 0 deletions ...rg/apache/lucene/backward_codecs/lucene102/TestLucene102BinaryQuantizedVectorsFormat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.KnnVectorValues;
import org.apache.lucene.index.LeafReader;
import org.apache.lucene.index.VectorEncoding;
import org.apache.lucene.index.VectorSimilarityFunction;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.KnnFloatVectorQuery;
Expand Down Expand Up @@ -187,6 +188,11 @@ public void testQuantizedVectorsWriteAndRead() throws IOException {
}
}

@Override
protected VectorEncoding randomVectorEncoding() {
return random().nextBoolean() ? VectorEncoding.BYTE : VectorEncoding.FLOAT32;
}

@Override
protected boolean supportsFloatVectorFallback() {
return false;
Expand Down
6 changes: 6 additions & 0 deletions ...pache/lucene/backward_codecs/lucene102/TestLucene102HnswBinaryQuantizedVectorsFormat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.KnnVectorValues;
import org.apache.lucene.index.LeafReader;
import org.apache.lucene.index.VectorEncoding;
import org.apache.lucene.index.VectorSimilarityFunction;
import org.apache.lucene.search.AcceptDocs;
import org.apache.lucene.search.TopDocs;
Expand Down Expand Up @@ -178,6 +179,11 @@ public void testSimpleOffHeapSize() throws IOException {
}
}

@Override
protected VectorEncoding randomVectorEncoding() {
return random().nextBoolean() ? VectorEncoding.BYTE : VectorEncoding.FLOAT32;
}

@Override
protected boolean supportsFloatVectorFallback() {
return false;
Expand Down
6 changes: 6 additions & 0 deletions ...codecs/src/test/org/apache/lucene/backward_codecs/lucene94/Lucene94HnswVectorsWriter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ private void writeField(FieldWriter<?> fieldData, int maxDoc) throws IOException
switch (fieldData.fieldInfo.getVectorEncoding()) {
case BYTE -> writeByteVectors(fieldData);
case FLOAT32 -> writeFloat32Vectors(fieldData);
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
}
long vectorDataLength = vectorData.getFilePointer() - vectorDataOffset;

Expand Down Expand Up @@ -241,6 +242,7 @@ private void writeSortingField(FieldWriter<?> fieldData, int maxDoc, Sorter.DocM
switch (fieldData.fieldInfo.getVectorEncoding()) {
case BYTE -> writeSortedByteVectors(fieldData, ordMap);
case FLOAT32 -> writeSortedFloat32Vectors(fieldData, ordMap);
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
};
long vectorDataLength = vectorData.getFilePointer() - vectorDataOffset;

Expand Down Expand Up @@ -405,6 +407,7 @@ public IORunnable mergeOneField(FieldInfo fieldInfo, MergeState mergeState) thro
writeVectorData(
tempVectorData,
MergedVectorValues.mergeFloatVectorValues(fieldInfo, mergeState));
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
};
CodecUtil.writeFooter(tempVectorData);
IOUtils.close(tempVectorData);
Expand Down Expand Up @@ -461,6 +464,7 @@ public IORunnable mergeOneField(FieldInfo fieldInfo, MergeState mergeState) thro
scorerSupplier, M, beamWidth, HnswGraphBuilder.randSeed);
yield hnswGraphBuilder.build(vectorValues.size());
}
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
};
writeGraph(graph);
}
Expand Down Expand Up @@ -662,6 +666,7 @@ public float[] copyValue(float[] value) {
return ArrayUtil.copyOfSubArray(value, 0, dim);
}
};
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
};
}

Expand All @@ -683,6 +688,7 @@ public float[] copyValue(float[] value) {
defaultFlatVectorScorer.getRandomVectorScorerSupplier(
fieldInfo.getVectorSimilarityFunction(),
FloatVectorValues.fromFloats((List<float[]>) vectors, dim));
case FLOAT16 -> throw new UnsupportedOperationException("FLOAT16 is not supported");
};
hnswGraphBuilder =
HnswGraphBuilder.create(scorerSupplier, M, beamWidth, HnswGraphBuilder.randSeed);
Expand Down
6 changes: 6 additions & 0 deletions ...cs/src/test/org/apache/lucene/backward_codecs/lucene94/TestLucene94HnswVectorsFormat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import org.apache.lucene.codecs.Codec;
import org.apache.lucene.codecs.KnnVectorsFormat;
import org.apache.lucene.index.VectorEncoding;
import org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase;

public class TestLucene94HnswVectorsFormat extends BaseKnnVectorsFormatTestCase {
Expand All @@ -43,4 +44,9 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
protected boolean supportsFloatVectorFallback() {
return false;
}

@Override
protected VectorEncoding randomVectorEncoding() {
return random().nextBoolean() ? VectorEncoding.BYTE : VectorEncoding.FLOAT32;
}
}
Loading
Loading