Commit 06087c4
committed
fix: AWS-CORS-safe
AWS S3 (and many other backends) strip `Content-Range` from CORS
responses by default. The old 1-byte range trick then fell back to the
partial `Content-Length` of 1, so hyparquet saw `byteLength = 1`, tried
to read the 8-byte footer, and tripped `RangeError: Offset is outside
the bounds of the DataView` on any public-S3-backed parquet.
Two changes:
- `asyncBufferFromStore`: when `store.getUrl` is defined, do an HTTP
HEAD first — its `Content-Length` is the full object size (200, not
206) and survives CORS. Fall back to the 1-byte trick for stores
without `getUrl` (native R2 binding, HTTP proxies that expose
`Content-Range`).
- `S3Store.get`: don't trust `Content-Length` as `totalSize` on a 206
response. Only set `totalSize` from `Content-Length` for 200s, or
from a present `Content-Range`. Avoids the same misread bleeding into
callers that don't use `asyncBufferFromStore`.
Reproed on `ctbk/stations/ids_201306:202209.parquet` (88 KB, 3257 rows);
now renders the full table.byteLength discovery in asyncBufferFromStore
1 parent 29f7829 commit 06087c4
2 files changed
Lines changed: 29 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
25 | 43 | | |
26 | 44 | | |
27 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
219 | 223 | | |
220 | 224 | | |
221 | | - | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
222 | 228 | | |
223 | 229 | | |
224 | 230 | | |
| |||
0 commit comments