Merged
Conversation
luciaquirke
commented
Oct 13, 2025
| @@ -0,0 +1,454 @@ | |||
| import os | |||
Collaborator
Author
There was a problem hiding this comment.
modified copy of build.py
luciaquirke
commented
Oct 13, 2025
luciaquirke
force-pushed
the
query-2
branch
2 times, most recently
from
0b2edc9 to
07f6af8
Compare
luciaquirke
commented
Oct 14, 2025
|
|
||
| # Asynchronously move the gradient to CPU and convert to fp16 | ||
| mod_grads[name] = g.to(device="cpu", dtype=dtype, non_blocking=True) | ||
| if save_index: |
Collaborator
Author
There was a problem hiding this comment.
Avoid the round trip to cpu
luciaquirke
commented
Oct 14, 2025
|
|
||
| precision: Literal["auto", "bf16", "fp16", "fp32", "int4", "int8"] = "auto" | ||
| """Precision to use for the model parameters.""" | ||
| """Precision (dtype) to use for the model parameters.""" |
Collaborator
Author
There was a problem hiding this comment.
improve searchability
luciaquirke
force-pushed
the
query-2
branch
3 times, most recently
from
9fe9bff to
353a1d7
Compare
luciaquirke
commented
Oct 16, 2025
| dtype=dtype, | ||
| fill_value=0.0, | ||
| ) | ||
| per_doc_scores = torch.full( |
Collaborator
Author
There was a problem hiding this comment.
Only support one score per doc, i.e. don't support computing module scores separately for now
luciaquirke
commented
Oct 16, 2025
| """Number of examples to use for estimating processor statistics.""" | ||
|
|
||
| drop_columns: bool = False | ||
| drop_columns: bool = True |
Collaborator
Author
There was a problem hiding this comment.
Prevent duplicating entire dataset on disk by default
…'t stream query dataset
luciaquirke
force-pushed
the
query-2
branch
3 times, most recently
from
6405510 to
9de0ecb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO
Notes
If keeping the extra gradients in VRAM before the query callback causes problems we can add something like
But it's only a small memory usage increase.