Commit b96e41e
authored
perf: Lazily load fast fields dictionaries. (#2842)
## What
Lazily load fast field dictionaries from buffers: see
paradedb/tantivy#55
## Why
A customer reported slower-than-expected paging on a string/uuid column.
85% of the time for that query was being spent in _opening_ a fast
fields string/bytes column, with a large fraction of that time spent
fully consuming the column's `Dictionary`.
## Tests
See the attached benchmark results:
* [`docs`
dataset](#2842 (review))
* No regressions.
* 2x faster for `top_n-score`
* 1.4x faster for `highlighting`
* [`logs`
dataset](#2842 (review))
* No regressions.
* 4.5x faster for `paging-string-max`
* 1.7x faster for `paging-string-median`
* 1.6x faster for `paging-string-min`
The `paging-string-*` benchmarks were added in #2834 to highlight this
particular issue.1 parent 5191c6a commit b96e41e
2 files changed
Lines changed: 11 additions & 11 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
0 commit comments