Parquet: support same-projection `RowFilter` fusion

[haohuaijin]

Is your feature request related to a problem or challenge?

RowFilter evaluates each ArrowPredicate separately, so same-projection predicates may decode or replay the same column more than once. #10859 reuses one decoded stream and evaluates later predicates only on surviving rows.

This can avoid repeated decoding or predicate-cache replay, but fragmented or dense survivors may be expensive to materialize. Fusion can therefore regress some workloads.

Describe the solution you'd like

When consecutive predicates use the same projection, the Parquet reader should be able to evaluate them from one decoded stream. This avoids repeated decoding or predicate-cache replay while preserving predicate order and ensuring that each predicate sees only rows accepted by earlier predicates.

Fusion should not be an unconditional rewrite: as described above, materializing survivors can cost more than the repeated work it removes for some execution shapes. How to decide when fusion applies — a conservative structural rule, caller configuration, or something smarter — is left open for discussion. #10859 currently uses a conservative structural gate (a single top-level, non-repeated leaf projection, and no active row limit).

Describe alternatives you've considered

No response

Additional context

• Arrow: perf(parquet): fuse consecutive filters on the same projection #10859, Improve performance of CachedArrayreader filter+concat #10774, parquet: Improve performance of mask/selection construction in ReadPlanBuilder::with_predicate_options #10776
• DataFusion: WIP: DO NOT REVIEW: Benchmark enabling parquet filter pushdown by default with one-shot row group I/O datafusion#24426, [TESTING] Benchmark same-projection Parquet filter fusion with pushdown enabled by default (arrow-rs#10859) datafusion#24705

[haohuaijin]

Benchmark results

• ClickBench Q25: 140.379 ms -> 110.727 ms with perf(parquet): fuse consecutive filters on the same projection #10859 (-21.12% latency, +26.78% throughput).
• Synthetic benchmark: 256 three-predicate cases over 4 million SNAPPY-compressed rows, with the filter column projected. Each query already had consecutive same-projection predicates, so DataFusion grouping had no effect.

All deltas below are relative to main pushdown ON; negative is faster.

Scope	Cases	#10859 vs main pushdown ON	Regressions above 2%
All	256	-6.84%	33
Int64, fragmented	64	-16.03%	10
Int64, clustered	64	+0.36%	20
String, fragmented	64	-9.74%	2
String, clustered	64	-0.96%	1

Fusion benefits selective-first cases, but can regress when most rows survive. For 99% -> 99% -> 99%, the regressions were 16.21% for fragmented Int64, 13.05% for clustered Int64, and 5.92% for fragmented strings. The best string case (20% -> 99% -> 1%, fragmented) improved by 23.59%.

Full per-case medians (ms); most cases use 15 samples and dedicated retests use 33:

Int64 / fragmented — 64 cases

Survivors	Output rows	main OFF (ms)	main ON (ms)	#10859 (ms)	vs ON	Samples
99% / 99% / 99%	3,881,196	43.746	52.566	61.088	+16.21%	15
99% / 99% / 50%	1,960,200	44.500	52.273	58.363	+11.65%	15
99% / 99% / 20%	784,080	43.634	50.707	55.298	+9.05%	15
99% / 99% / 1%	39,204	42.148	49.238	51.462	+4.52%	15
99% / 50% / 99%	1,960,200	44.626	49.351	52.841	+7.07%	15
99% / 50% / 50%	990,000	43.913	48.312	51.172	+5.92%	15
99% / 50% / 20%	396,000	42.989	47.034	49.218	+4.64%	15
99% / 50% / 1%	19,800	42.433	46.438	47.490	+2.27%	15
99% / 20% / 99%	784,080	41.056	44.407	45.369	+2.17%	15
99% / 20% / 50%	396,000	41.318	42.621	44.142	+3.57%	33
99% / 20% / 20%	158,400	41.075	43.453	43.224	-0.53%	15
99% / 20% / 1%	7,920	40.614	42.901	42.566	-0.78%	15
99% / 1% / 99%	39,204	36.717	41.121	39.950	-2.85%	15
99% / 1% / 50%	19,800	36.730	39.682	39.309	-0.94%	33
99% / 1% / 20%	7,920	36.560	40.635	39.380	-3.09%	15
99% / 1% / 1%	396	36.909	40.410	39.276	-2.81%	15
50% / 99% / 99%	1,960,200	44.899	72.755	57.631	-20.79%	15
50% / 99% / 50%	990,000	43.569	70.303	53.304	-24.18%	15
50% / 99% / 20%	396,000	42.663	68.755	49.577	-27.89%	15
50% / 99% / 1%	19,800	42.194	64.577	44.165	-31.61%	15
50% / 50% / 99%	990,000	43.159	63.271	50.066	-20.87%	15
50% / 50% / 50%	500,000	42.981	62.646	47.497	-24.18%	15
50% / 50% / 20%	200,000	43.110	61.650	46.149	-25.14%	15
50% / 50% / 1%	10,000	42.247	58.145	41.738	-28.22%	15
50% / 20% / 99%	396,000	39.361	58.411	44.340	-24.09%	15
50% / 20% / 50%	200,000	39.480	58.505	43.535	-25.59%	15
50% / 20% / 20%	80,000	39.415	53.328	38.648	-27.53%	33
50% / 20% / 1%	4,000	39.303	55.502	38.564	-30.52%	15
50% / 1% / 99%	19,800	36.528	49.269	37.275	-24.34%	15
50% / 1% / 50%	10,000	36.504	48.893	36.880	-24.57%	15
50% / 1% / 20%	4,000	36.582	48.563	36.696	-24.44%	15
50% / 1% / 1%	200	36.349	48.684	36.529	-24.97%	15
20% / 99% / 99%	784,080	46.703	65.644	47.516	-27.62%	15
20% / 99% / 50%	396,000	46.999	64.039	44.273	-30.87%	15
20% / 99% / 20%	158,400	47.210	63.061	41.524	-34.15%	15
20% / 99% / 1%	7,920	46.432	58.473	36.553	-37.49%	15
20% / 50% / 99%	396,000	43.210	59.408	42.486	-28.48%	15
20% / 50% / 50%	200,000	43.209	58.749	40.623	-30.85%	15
20% / 50% / 20%	80,000	43.163	57.400	39.710	-30.82%	15
20% / 50% / 1%	4,000	42.857	53.361	34.931	-34.54%	15
20% / 20% / 99%	158,400	40.214	54.945	39.367	-28.35%	15
20% / 20% / 50%	80,000	40.474	55.012	38.847	-29.38%	15
20% / 20% / 20%	32,000	40.618	51.430	35.047	-31.85%	15
20% / 20% / 1%	1,600	40.462	50.246	34.343	-31.65%	15
20% / 1% / 99%	7,920	38.741	46.010	33.698	-26.76%	15
20% / 1% / 50%	4,000	38.856	45.892	33.625	-26.73%	15
20% / 1% / 20%	1,600	38.778	45.788	32.796	-28.38%	15
20% / 1% / 1%	80	38.555	45.733	32.763	-28.36%	15
1% / 99% / 99%	39,204	31.786	36.002	33.230	-7.70%	15
1% / 99% / 50%	19,800	31.797	35.668	32.602	-8.60%	15
1% / 99% / 20%	7,920	31.745	35.020	31.868	-9.00%	15
1% / 99% / 1%	396	31.908	34.579	31.351	-9.34%	15
1% / 50% / 99%	19,800	31.403	34.844	32.449	-6.88%	15
1% / 50% / 50%	10,000	31.489	33.416	31.073	-7.01%	33
1% / 50% / 20%	4,000	31.311	33.946	31.664	-6.72%	15
1% / 50% / 1%	200	31.629	33.975	31.319	-7.82%	15
1% / 20% / 99%	7,920	31.123	34.059	31.771	-6.72%	15
1% / 20% / 50%	4,000	31.123	33.745	31.723	-5.99%	15
1% / 20% / 20%	1,600	30.967	33.612	31.418	-6.53%	15
1% / 20% / 1%	80	31.200	33.318	31.089	-6.69%	15
1% / 1% / 99%	396	31.186	33.025	31.017	-6.08%	15
1% / 1% / 50%	200	31.009	32.342	30.587	-5.43%	33
1% / 1% / 20%	80	31.053	32.825	31.207	-4.93%	15
1% / 1% / 1%	4	30.978	33.247	31.071	-6.55%	15

Int64 / clustered — 64 cases

Survivors	Output rows	main OFF (ms)	main ON (ms)	#10859 (ms)	vs ON	Samples
99% / 99% / 99%	3,881,196	28.026	32.096	36.284	+13.05%	15
99% / 99% / 50%	1,960,200	27.509	31.175	35.154	+12.77%	15
99% / 99% / 20%	784,080	27.465	30.734	34.806	+13.25%	15
99% / 99% / 1%	39,204	26.795	30.345	34.587	+13.98%	15
99% / 50% / 99%	1,960,200	29.370	30.231	32.633	+7.95%	15
99% / 50% / 50%	990,000	28.014	29.191	31.446	+7.73%	15
99% / 50% / 20%	396,000	27.638	28.427	30.620	+7.72%	15
99% / 50% / 1%	19,800	26.918	28.254	30.122	+6.61%	15
99% / 20% / 99%	784,080	26.136	26.159	27.095	+3.58%	15
99% / 20% / 50%	396,000	25.933	25.524	26.219	+2.72%	15
99% / 20% / 20%	158,400	25.565	25.528	26.111	+2.28%	15
99% / 20% / 1%	7,920	25.356	25.336	26.171	+3.30%	15
99% / 1% / 99%	39,204	21.406	23.185	23.430	+1.05%	15
99% / 1% / 50%	19,800	21.479	23.187	23.317	+0.56%	15
99% / 1% / 20%	7,920	21.404	23.061	23.287	+0.98%	15
99% / 1% / 1%	396	21.521	22.898	23.220	+1.41%	15
50% / 99% / 99%	1,960,200	29.310	28.624	29.326	+2.45%	15
50% / 99% / 50%	990,000	28.494	27.599	28.205	+2.20%	15
50% / 99% / 20%	396,000	27.727	26.527	27.098	+2.15%	15
50% / 99% / 1%	19,800	26.788	26.064	26.769	+2.70%	15
50% / 50% / 99%	990,000	27.717	25.136	26.026	+3.54%	15
50% / 50% / 50%	500,000	27.054	24.518	25.440	+3.76%	15
50% / 50% / 20%	200,000	26.632	24.099	24.769	+2.78%	15
50% / 50% / 1%	10,000	26.629	23.901	24.389	+2.04%	15
50% / 20% / 99%	396,000	22.656	22.589	22.608	+0.08%	15
50% / 20% / 50%	200,000	22.382	22.395	22.408	+0.06%	15
50% / 20% / 20%	80,000	22.260	22.328	22.349	+0.10%	15
50% / 20% / 1%	4,000	22.190	22.448	22.055	-1.75%	15
50% / 1% / 99%	19,800	21.273	21.561	20.892	-3.10%	15
50% / 1% / 50%	10,000	21.252	21.112	21.029	-0.39%	15
50% / 1% / 20%	4,000	21.459	21.213	20.972	-1.14%	15
50% / 1% / 1%	200	21.242	20.978	20.653	-1.55%	15
20% / 99% / 99%	784,080	23.803	21.320	21.215	-0.49%	15
20% / 99% / 50%	396,000	22.969	20.706	20.497	-1.01%	15
20% / 99% / 20%	158,400	22.884	20.507	20.286	-1.08%	15
20% / 99% / 1%	7,920	22.560	20.165	19.777	-1.92%	15
20% / 50% / 99%	396,000	22.407	20.217	19.703	-2.54%	15
20% / 50% / 50%	200,000	21.812	19.881	19.386	-2.49%	15
20% / 50% / 20%	80,000	21.778	19.500	19.087	-2.12%	15
20% / 50% / 1%	4,000	21.832	19.440	18.908	-2.74%	15
20% / 20% / 99%	158,400	21.524	18.948	18.397	-2.91%	15
20% / 20% / 50%	80,000	21.160	18.729	18.238	-2.62%	15
20% / 20% / 20%	32,000	21.305	18.215	18.310	+0.52%	15
20% / 20% / 1%	1,600	21.209	18.539	18.713	+0.94%	15
20% / 1% / 99%	7,920	20.866	17.529	16.942	-3.35%	33
20% / 1% / 50%	4,000	20.950	18.038	17.777	-1.45%	15
20% / 1% / 20%	1,600	20.972	17.916	17.419	-2.77%	15
20% / 1% / 1%	80	20.979	17.920	17.573	-1.94%	15
1% / 99% / 99%	39,204	15.795	17.093	16.482	-3.57%	15
1% / 99% / 50%	19,800	15.625	16.804	16.032	-4.59%	15
1% / 99% / 20%	7,920	15.651	16.796	15.945	-5.07%	15
1% / 99% / 1%	396	15.695	16.636	15.846	-4.75%	15
1% / 50% / 99%	19,800	15.539	16.703	16.251	-2.71%	15
1% / 50% / 50%	10,000	15.652	16.714	16.161	-3.31%	15
1% / 50% / 20%	4,000	15.544	16.439	15.908	-3.23%	15
1% / 50% / 1%	200	15.632	16.384	15.859	-3.21%	15
1% / 20% / 99%	7,920	15.605	16.506	15.860	-3.92%	15
1% / 20% / 50%	4,000	15.505	16.528	15.956	-3.46%	15
1% / 20% / 20%	1,600	15.459	16.365	15.749	-3.76%	15
1% / 20% / 1%	80	15.561	16.115	15.732	-2.38%	15
1% / 1% / 99%	396	15.592	16.211	15.752	-2.83%	15
1% / 1% / 50%	200	15.484	16.310	15.823	-2.98%	15
1% / 1% / 20%	80	15.455	16.126	15.781	-2.14%	15
1% / 1% / 1%	4	15.541	16.176	15.602	-3.55%	15

String / fragmented — 64 cases

Survivors	Output rows	main OFF (ms)	main ON (ms)	#10859 (ms)	vs ON	Samples
99% / 99% / 99%	3,881,196	117.890	132.586	140.436	+5.92%	15
99% / 99% / 50%	1,960,200	124.063	131.813	137.643	+4.42%	15
99% / 99% / 20%	784,080	128.852	138.710	141.145	+1.76%	15
99% / 99% / 1%	39,204	123.738	137.830	140.219	+1.73%	15
99% / 50% / 99%	1,960,200	129.916	122.287	123.395	+0.91%	15
99% / 50% / 50%	990,000	121.564	120.055	119.547	-0.42%	15
99% / 50% / 20%	396,000	123.792	122.099	121.964	-0.11%	15
99% / 50% / 1%	19,800	121.836	121.306	120.879	-0.35%	15
99% / 20% / 99%	784,080	106.768	110.803	111.303	+0.45%	15
99% / 20% / 50%	396,000	102.987	109.854	109.984	+0.12%	15
99% / 20% / 20%	158,400	102.592	110.972	110.199	-0.70%	15
99% / 20% / 1%	7,920	100.084	110.845	109.668	-1.06%	15
99% / 1% / 99%	39,204	90.897	101.076	99.545	-1.51%	15
99% / 1% / 50%	19,800	90.153	101.288	98.992	-2.27%	15
99% / 1% / 20%	7,920	91.036	100.841	99.375	-1.45%	15
99% / 1% / 1%	396	90.522	101.152	98.823	-2.30%	15
50% / 99% / 99%	1,960,200	133.292	142.538	123.237	-13.54%	15
50% / 99% / 50%	990,000	124.118	138.827	117.956	-15.03%	15
50% / 99% / 20%	396,000	127.281	141.876	118.392	-16.55%	15
50% / 99% / 1%	19,800	124.229	137.284	112.891	-17.77%	15
50% / 50% / 99%	990,000	124.555	129.143	111.699	-13.51%	15
50% / 50% / 50%	500,000	119.285	127.331	108.335	-14.92%	15
50% / 50% / 20%	200,000	124.038	129.332	108.548	-16.07%	15
50% / 50% / 1%	10,000	122.229	123.983	102.586	-17.26%	15
50% / 20% / 99%	396,000	100.972	122.344	105.001	-14.18%	15
50% / 20% / 50%	200,000	99.061	122.537	102.325	-16.49%	15
50% / 20% / 20%	80,000	99.152	118.817	98.350	-17.23%	15
50% / 20% / 1%	4,000	98.690	117.867	97.244	-17.50%	15
50% / 1% / 99%	19,800	92.577	105.816	90.285	-14.68%	15
50% / 1% / 50%	10,000	92.410	104.975	89.755	-14.50%	15
50% / 1% / 20%	4,000	92.091	105.507	89.874	-14.82%	15
50% / 1% / 1%	200	91.909	105.525	90.638	-14.11%	15
20% / 99% / 99%	784,080	106.364	127.511	103.501	-18.83%	15
20% / 99% / 50%	396,000	103.675	125.120	99.524	-20.46%	15
20% / 99% / 20%	158,400	103.101	125.179	97.971	-21.74%	15
20% / 99% / 1%	7,920	100.773	115.770	88.461	-23.59%	33
20% / 50% / 99%	396,000	98.537	119.060	97.837	-17.83%	15
20% / 50% / 50%	200,000	96.042	117.479	95.898	-18.37%	15
20% / 50% / 20%	80,000	95.890	117.946	95.633	-18.92%	15
20% / 50% / 1%	4,000	95.177	112.391	88.214	-21.51%	15
20% / 20% / 99%	158,400	91.744	112.561	93.054	-17.33%	15
20% / 20% / 50%	80,000	91.087	112.553	91.615	-18.60%	15
20% / 20% / 20%	32,000	90.706	108.781	87.328	-19.72%	15
20% / 20% / 1%	1,600	90.505	107.961	85.905	-20.43%	15
20% / 1% / 99%	7,920	87.246	99.848	82.741	-17.13%	15
20% / 1% / 50%	4,000	87.046	99.423	83.404	-16.11%	15
20% / 1% / 20%	1,600	86.376	98.748	82.624	-16.33%	15
20% / 1% / 1%	80	86.532	99.222	82.758	-16.59%	15
1% / 99% / 99%	39,204	85.522	94.378	90.988	-3.59%	15
1% / 99% / 50%	19,800	85.408	93.829	90.262	-3.80%	15
1% / 99% / 20%	7,920	85.235	93.113	89.694	-3.67%	15
1% / 99% / 1%	396	84.677	93.570	89.449	-4.40%	15
1% / 50% / 99%	19,800	84.981	93.234	90.367	-3.08%	15
1% / 50% / 50%	10,000	85.279	92.752	89.799	-3.18%	15
1% / 50% / 20%	4,000	85.336	92.460	90.016	-2.64%	15
1% / 50% / 1%	200	84.628	92.645	89.650	-3.23%	15
1% / 20% / 99%	7,920	84.090	92.807	89.374	-3.70%	15
1% / 20% / 50%	4,000	84.790	91.980	89.141	-3.09%	15
1% / 20% / 20%	1,600	84.220	92.138	88.724	-3.70%	15
1% / 20% / 1%	80	84.662	92.367	88.648	-4.03%	15
1% / 1% / 99%	396	83.895	91.131	88.839	-2.52%	15
1% / 1% / 50%	200	83.894	90.870	88.704	-2.38%	15
1% / 1% / 20%	80	84.408	91.558	88.893	-2.91%	15
1% / 1% / 1%	4	83.756	89.496	86.913	-2.89%	15

String / clustered — 64 cases

Survivors	Output rows	main OFF (ms)	main ON (ms)	#10859 (ms)	vs ON	Samples
99% / 99% / 99%	3,881,196	99.077	110.325	112.492	+1.96%	15
99% / 99% / 50%	1,960,200	97.024	106.444	108.291	+1.74%	15
99% / 99% / 20%	784,080	102.954	111.539	114.956	+3.06%	15
99% / 99% / 1%	39,204	102.894	111.331	113.502	+1.95%	15
99% / 50% / 99%	1,960,200	108.491	99.797	99.584	-0.21%	15
99% / 50% / 50%	990,000	101.683	97.040	96.839	-0.21%	15
99% / 50% / 20%	396,000	103.752	99.334	99.290	-0.04%	15
99% / 50% / 1%	19,800	100.308	97.943	97.804	-0.14%	15
99% / 20% / 99%	784,080	95.323	90.828	89.761	-1.17%	15
99% / 20% / 50%	396,000	91.438	89.359	87.258	-2.35%	15
99% / 20% / 20%	158,400	93.170	89.280	87.697	-1.77%	15
99% / 20% / 1%	7,920	92.267	88.386	86.858	-1.73%	15
99% / 1% / 99%	39,204	73.819	81.634	80.286	-1.65%	15
99% / 1% / 50%	19,800	73.254	81.349	79.185	-2.66%	15
99% / 1% / 20%	7,920	73.944	80.511	78.568	-2.41%	15
99% / 1% / 1%	396	73.327	79.905	78.250	-2.07%	15
50% / 99% / 99%	1,960,200	113.581	92.253	90.741	-1.64%	15
50% / 99% / 50%	990,000	104.936	89.151	88.002	-1.29%	15
50% / 99% / 20%	396,000	106.827	92.123	90.634	-1.62%	15
50% / 99% / 1%	19,800	103.270	90.485	89.405	-1.19%	15
50% / 50% / 99%	990,000	102.350	84.748	84.340	-0.48%	15
50% / 50% / 50%	500,000	97.922	82.815	82.418	-0.48%	15
50% / 50% / 20%	200,000	100.446	84.678	83.497	-1.39%	15
50% / 50% / 1%	10,000	99.101	82.461	82.220	-0.29%	15
50% / 20% / 99%	396,000	82.332	79.004	78.037	-1.22%	15
50% / 20% / 50%	200,000	80.429	77.597	76.264	-1.72%	15
50% / 20% / 20%	80,000	79.445	77.653	77.202	-0.58%	15
50% / 20% / 1%	4,000	79.411	77.391	76.180	-1.57%	15
50% / 1% / 99%	19,800	74.695	74.140	73.036	-1.49%	15
50% / 1% / 50%	10,000	73.898	70.531	69.285	-1.77%	33
50% / 1% / 20%	4,000	73.758	72.988	71.611	-1.89%	15
50% / 1% / 1%	200	73.548	72.596	70.979	-2.23%	15
20% / 99% / 99%	784,080	86.128	75.477	73.172	-3.05%	15
20% / 99% / 50%	396,000	82.678	73.499	71.042	-3.34%	15
20% / 99% / 20%	158,400	81.763	73.371	72.027	-1.83%	15
20% / 99% / 1%	7,920	80.045	73.040	71.355	-2.31%	15
20% / 50% / 99%	396,000	79.991	71.512	69.436	-2.90%	15
20% / 50% / 50%	200,000	77.390	69.659	68.913	-1.07%	15
20% / 50% / 20%	80,000	77.366	69.870	69.563	-0.44%	15
20% / 50% / 1%	4,000	77.048	69.521	68.921	-0.86%	15
20% / 20% / 99%	158,400	75.975	68.914	68.603	-0.45%	15
20% / 20% / 50%	80,000	75.064	67.471	66.996	-0.70%	15
20% / 20% / 20%	32,000	74.932	67.514	66.688	-1.22%	15
20% / 20% / 1%	1,600	75.041	67.166	66.694	-0.70%	15
20% / 1% / 99%	7,920	73.156	66.708	65.704	-1.50%	15
20% / 1% / 50%	4,000	72.497	65.909	65.480	-0.65%	15
20% / 1% / 20%	1,600	71.653	65.140	64.665	-0.73%	15
20% / 1% / 1%	80	71.872	65.548	64.255	-1.97%	15
1% / 99% / 99%	39,204	67.239	73.180	72.102	-1.47%	15
1% / 99% / 50%	19,800	67.319	72.418	71.655	-1.05%	15
1% / 99% / 20%	7,920	66.880	71.880	71.036	-1.17%	15
1% / 99% / 1%	396	67.275	71.375	70.574	-1.12%	15
1% / 50% / 99%	19,800	66.131	72.569	72.404	-0.23%	15
1% / 50% / 50%	10,000	66.967	71.670	70.992	-0.95%	15
1% / 50% / 20%	4,000	66.900	71.300	71.873	+0.80%	15
1% / 50% / 1%	200	67.004	71.008	70.195	-1.14%	15
1% / 20% / 99%	7,920	66.908	72.498	72.130	-0.51%	15
1% / 20% / 50%	4,000	66.373	71.664	71.709	+0.06%	15
1% / 20% / 20%	1,600	66.760	68.750	68.241	-0.74%	33
1% / 20% / 1%	80	66.747	70.190	69.985	-0.29%	15
1% / 1% / 99%	396	66.511	72.250	71.745	-0.70%	15
1% / 1% / 50%	200	66.545	71.259	70.455	-1.13%	15
1% / 1% / 20%	80	66.385	70.703	70.921	+0.31%	15
1% / 1% / 1%	4	67.187	70.051	70.298	+0.35%	15