Preconditioning #137997
Draft
Preconditioning #137997
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ctors to become indistinguishable
benwtrent
reviewed
Nov 13, 2025
| final FloatVectorValues floatVectorValues = getFloatVectorValues(fieldWriter.fieldInfo, fieldWriter.delegate, maxDoc); | ||
| FloatVectorValues floatVectorValues = getFloatVectorValues(fieldWriter.fieldInfo, fieldWriter.delegate, maxDoc); | ||
| // precondition the vectors if necessary | ||
| floatVectorValues = preconditionVectors(floatVectorValues); |
Member
There was a problem hiding this comment.
I wonder if we can precondition just immediately before quantization and precondition the centroids after clustering. Maybe applying it before clustering works ok...I need to think about that.
Contributor
Author
There was a problem hiding this comment.
tried both; seems to work fine to precondition before creating the centroids; still worth thinking about though. I will post numbers for review.
Contributor
Author
|
Ran on a variety of datasets so we could see the impact of preconditioning on QPS in both scenarios where we expect to see improvements and scenarios where we don't. My quick assessment from running these is we generally seem to get the best result out of using the variance-based permutation matrix. For equivalent recall it has reasonable QPS. And in general when attempting to recover recall there are some dataset where it's absolutely necessary to use preconditioning. It would seem this is valuable even at a segment level imo. Thinking that it makes sense to follow up here by investigating if I can sample the average variances across dimensions as a mechanism to determine when to use preconditioning automatically. Below are all the discussed datasets except for mnist. dbpediacohereglove 100dhotpotqamulti_towerfashion mnistgistsift |
Member
I realize that this is likely best, in that its applying the permutation per segment already. But the huge benefit of random is that it can be applied once, globally for the query vector and conceivably random permutation will be much cheaper at index and merge as the permutation matrix doesn't need to evolve with the vector distributions, and instead is static. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial pass at a preconditioning utils and wiring through for the disk bbq next format so we can iterate to see it's impact on quantization.