🐛 Fix local file parsing in gguf package #1918
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix off-by-one error in RangeViewLocalFile.fetchChunk: Blob.slice uses exclusive end, but range was calculated for inclusive end - Fix FileBlob.slice to clamp start position to file end, preventing negative size when slicing beyond EOF (affects small/sharded files)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Fixes #1915 -
gguf()fails withArrayBuffer.prototype.resize: Invalid length parameterwhen parsing GGUF files usingallowLocalFile: true.Two bugs were identified:
Off-by-one error in chunk fetching:
RangeViewLocalFile.fetchChunk()calculates range as[start, end-1](inclusive end, matching HTTP Range headers), but passes it directly toBlob.slice(start, end)which uses an exclusive end. This causes each chunk to be 1 byte short, creating gaps at chunk boundaries that break metadata parsing.Negative slice size for small files:
FileBlob.slice()could create slices with negative size when the requested range starts beyond EOF. This affects small files and sharded GGUF files where the file is smaller than the 2MB chunk size.How does it solve it?
fetchChunk()to pass the correct exclusive end toBlob.slice()FileBlob.slice()to ensure start position never exceeds file endBreaking changes
None - this is a bugfix that makes local file parsing work correctly.