Buffer up to 2048 doc ids in for_each_docset_buffered - #2944
Draft
fulmicoton-dd wants to merge 1 commit into
Draft
Buffer up to 2048 doc ids in for_each_docset_buffered#2944fulmicoton-dd wants to merge 1 commit into
fulmicoton-dd wants to merge 1 commit into
Conversation
The no-score collection path (Weight::for_each_no_score) handed the collector's collect_block one COLLECT_BLOCK_BUFFER_LEN (64) block at a time. For aggregations this is the dominant path, and 64 docs per collect_block under-amortizes the per-call overhead. for_each_docset_buffered now owns a 2048-element heap buffer and fills it through successive fill_buffer calls over 64-element windows, flushing a single larger block to collect_block. fill_buffer keeps its 64-element window contract, so no DocSet implementation changes. The buffer is allocated with Box::new_zeroed_slice (stable since 1.92, hence the MSRV bump) to zero directly on the heap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
In the no-score collection path (
Weight::for_each_no_score→for_each_docset_buffered),matched doc ids were handed to the collector's
collect_blockoneCOLLECT_BLOCK_BUFFER_LEN(64) block at a time. For aggregations this is the dominant path(
requires_scoring() == false), and 64 docs percollect_blockcall under-amortizes theper-call overhead (virtual dispatch + aggregation bookkeeping).
for_each_docset_bufferednow owns a 2048-element heap buffer and fills it throughsuccessive
fill_buffercalls over 64-element windows, flushing a single larger block tocollect_block.fill_bufferkeeps itsCOLLECT_BLOCK_BUFFER_LEN(64) window contract, sono
DocSetimplementation changes — only how much we accumulate before flushing.Notes
Box::new_zeroed_slice(..).assume_init()to zero directlyon the heap (no 2048-element stack temporary). This API is stable since 1.92.0, hence the
MSRV bump
1.86 → 1.92.collect_blockimplementation: none assume a max slice length of 64 (theydelegate or iterate the slice; filter wrappers use a growable
Vec). Doc comments onCOLLECT_BLOCK_BUFFER_LENandSegmentCollector::collect_blockupdated accordingly — theblock handed to
collect_blockmay now exceed 64.unsafeto reinterpret each 64-elementchunks_exact_mutwindow as&mut [DocId; 64]for the
fill_buffercall (sound: 2048 is a multiple of 64).Benchmarks
cargo bench --bench agg_benchon thefull(1M-doc) input, single segment, no deletes — so every bench exercises thefor_each_no_score→collect_blockpath. Reported value is the minimum of 3 runs (most stable estimator; the per-run median swung ±5–20% on its own). Apple Silicon (M-series),REUSE_AGG_BENCH_INDEXso both sides hit the identical index.Summary: median -1.7%, mean -1.6% across 47 aggregations. Negative = faster. Broad improvement on the cheap full-scan aggregations (where per-
collect_blockoverhead dominated); a handful of heavyorder_by_card/top_hitsbenches regress and are within this harness's run-to-run noise.Per-benchmark median (min of 3), sorted by delta
histogram_hard_boundscardinality_agg_low_cardfilter_agg_all_query_count_agghistogramterms_many_order_by_termrange_agg_with_term_agg_statusfilter_agg_term_query_count_aggaverage_u64average_f64average_f64_u64range_aggrange_agg_with_avg_sub_aggterms_status_with_cardinality_aggterms_all_uniqueavg_and_range_with_avg_sub_aggterms_all_unique_with_avg_sub_aggterms_status_with_avg_sub_aggstats_f64filter_agg_all_query_with_sub_aggsextendedstats_f64histogram_with_avg_sub_agghistogram_with_term_agg_statusterms_7filter_agg_term_query_with_sub_aggsterms_150_000range_agg_with_term_agg_manypercentiles_f64terms_status_with_terms_zipf_1000_sub_aggterms_status_with_histogramcardinality_aggterms_many_with_avg_sub_aggcardinality_agg_high_cardterms_many_top_1000composite_term_many_page_1000composite_term_many_page_1000_with_avg_sub_aggterms_zipf_1000_with_terms_status_sub_aggterms_100_buckets_with_cardinality_aggterms_zipf_1000_with_histogramcomposite_histogram_calendarcomposite_term_fewterms_zipf_1000_with_avg_sub_aggterms_many_json_mixed_type_with_avg_sub_aggcomposite_histogramterms_many_with_top_hitsterms_zipf_1000terms_many_with_single_term_2_order_by_cardterms_many_with_single_term_order_by_cardTests
cargo build,cargo clippy --all-targets(no new warnings), andcargo test --lib {aggregation, query::, collector::}all pass.