[Feature](lambda) Support some map lambda functions

[linrrzqqq]

What problem does this PR solve?

Related PR: #67045, #67047

Doc: apache/doris-website#4085

Problem Summary:

This PR adds MAP lambda support under the Nereids planner.

Supported higher-order functions:

• map_filter((k, v) -> predicate, map)
• map_exists((k, v) -> predicate, map)
• map_all((k, v) -> predicate, map)
• map_apply((k, v) -> struct(new_key, new_value), map)
• transform_keys((k, v) -> new_key, map)
• transform_values((k, v) -> new_value, map)

Example:

SELECT map_filter(
    (k, v) -> v > 10,
    map(1, 10, 2, 20)
);

SELECT transform_values(
    (k, v) -> v + 1,
    map(1, 10, 2, 20)
);

SELECT map_apply(
    (k, v) -> (k + 1, v * 2),
    map(1, 10, 2, 20)
);

Implementation

Use map_entries(m) to expand the map parameter and reuse the original array lambda for execution.

map_apply((k, v) -> (k2, v2), m)
    ↓
map_from_entries(
    array_apply(entry: struct{k, v} -> (k2, v2), map_entries(m)
);

map_filter((k, v) -> k + v > 0, m)
    ↓
%map_from_filtered_entries_unique%(
    array_map(
        entry: strct{k, v} -> if(
            e.k + e.v > 0, entry, null
        ),
    map_entries(m)
)


transform_values((k, v) -> k + v + col, m)
    ↓
%map_from_entries_unique%(
    array_map(
        entry -> (e.k, e.k + e.v + col),
        map_entries(m)
    )
)


map_exists((k, v) -> k + v = 1, m)
    ↓
array_match_any(
    array_map(
        entry -> e.k + e.v = 1,
        map_entries(m)
    )
)


map_all((k, v) -> k + v = 1, m)
    ↓
array_match_all(
    array_map(
        entry -> e.k + e.v = 1,
        map_entries(m)
    )
)

Internal helper function

This PR adds two internal Map construction functions used by the rewritten expressions:

• %map_from_entries_unique%: rebuilds a Map from an entry array without running key deduplication. Used by transform_value
• %map_from_filtered_entries_unique%: removes null entries input instead of throw error like map_from_entries. Used by map_filter

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[linrrzqqq]

/review

[github-actions]

Automated review status: complete and converged after three rounds. Requesting changes for three P1 correctness issues.

Findings

1. The FE materialization rule moves sensitive Map inputs from Join candidate-pair scope into a child-row Project, changing outer-join error behavior and volatile evaluation frequency.
2. The same rule moves computed Maps out of selector-controlled IF/CASE branches, defeating supported short-circuit execution.
3. The BE eight-batch direct path bypasses the variable-width lambda safety ceiling and can turn a bounded query into a multi-gigabyte ColumnString overflow.

Checkpoint conclusions

• Goal and scope: the advertised Map lambda wrappers, constructors, tuple syntax, recursive NULL-type handling, and FE-to-BE lowerings are present. The scalar wrappers and registrations are localized; the 786-line late materialization rule and the unrelated ArrayMap batching expansion are the principal risk surfaces.
• Planner semantics and parallel paths: Project, OneRowRelation, Filter, Having, normalized Aggregate, Generate, Join metadata, nested lambda ExprIds, marker translation, and second-pass stability were reviewed. Outside the two inline evaluation-domain failures, no additional owner or parallel path defect survived.
• Types, nullability, compatibility, and physical symbols: lambda arity, Map-key legality, nullable Maps/predicates, nested NULL_TYPE merging, last-win semantics, and the names map_from_arrays, %map_from_arrays_unique%, map_filter, and map_from_entries align across FE and BE. No serialized format or existing function signature changes.
• Runtime correctness and ownership: constant/nullable wrappers, offsets, selected rows, invalid entries, duplicate keys, COW detachment, and Status propagation were checked. Those paths remain sound outside the accepted lambda budget issue.
• Performance and memory: the direct-path multiplier is a correctness failure as well as an allocation regression; no other distinct performance defect survived review.
• Concurrency, lifecycle, configuration, persistence, transactions, writes, and observability: no new applicable mechanism is introduced.
• Tests: the changed unit and regression sources broadly cover ordinary, null, constant, duplicate, empty, nested, aggregate, Join, Generate, and selected-row behavior, but omit the three accepted boundaries. Per the automated-review contract, I did not run builds or tests; current style, license, formatting, and secret checks pass, while build/test jobs are skipped.
• User focus: no additional focus was supplied.

No pre-existing live review thread or comment duplicated these findings.

[linrrzqqq]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 17554 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2130e12f16144fc369125d34a5dda5309b3e7184, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17629	3094	3081	3081
q2	1928	255	154	154
q3	10410	954	562	562
q4	4678	264	218	218
q5	7642	580	410	410
q6	139	116	97	97
q7	510	513	399	399
q8	9242	894	1022	894
q9	3539	2462	2432	2432
q10	6528	861	720	720
q11	463	258	234	234
q12	696	388	330	330
q13	17883	1539	1200	1200
q14	155	149	144	144
q15	q16	445	402	363	363
q17	842	733	838	733
q18	3337	2268	2289	2268
q19	1104	941	838	838
q20	662	503	478	478
q21	5301	1765	1935	1765
q22	330	275	234	234
Total cold run time: 93463 ms
Total hot run time: 17554 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3444	3346	3363	3346
q2	205	223	153	153
q3	2363	2475	2245	2245
q4	1224	1203	921	921
q5	2287	2141	2145	2141
q6	184	131	89	89
q7	1073	977	892	892
q8	1659	1461	1461	1461
q9	3223	3222	3242	3222
q10	1908	1883	1669	1669
q11	363	276	262	262
q12	463	434	340	340
q13	1515	1571	1225	1225
q14	195	187	165	165
q15	q16	397	405	374	374
q17	1075	1032	1037	1032
q18	5215	4590	5136	4590
q19	898	895	887	887
q20	1106	1039	1105	1039
q21	3912	3195	3252	3195
q22	413	361	317	317
Total cold run time: 33122 ms
Total hot run time: 29565 ms

[hello-stephen]

TPC-DS: Total hot run time: 84739 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2130e12f16144fc369125d34a5dda5309b3e7184, data reload: false

query5	4265	425	333	333
query6	430	162	148	148
query7	4856	460	272	272
query8	317	122	116	116
query9	8693	2890	2918	2890
query10	451	260	228	228
query11	5386	1066	924	924
query12	119	69	70	69
query13	1198	461	310	310
query14	6020	2253	2139	2139
query14_1	2009	1974	2016	1974
query15	172	122	112	112
query16	919	397	375	375
query17	795	459	370	370
query18	2338	329	239	239
query19	171	140	124	124
query20	73	75	69	69
query21	211	119	103	103
query22	5855	5395	5650	5395
query23	6869	6376	6083	6083
query23_1	6075	6220	6269	6220
query24	7296	1111	810	810
query24_1	807	790	820	790
query25	447	323	274	274
query26	1256	274	181	181
query27	2712	458	295	295
query28	4592	1498	1496	1496
query29	921	435	350	350
query30	272	182	177	177
query31	871	461	392	392
query32	101	50	51	50
query33	459	218	181	181
query34	1016	836	496	496
query35	399	398	352	352
query36	574	542	536	536
query37	116	82	69	69
query38	1016	848	835	835
query39	529	482	476	476
query39_1	501	473	450	450
query40	220	127	110	110
query41	54	52	50	50
query42	82	79	83	79
query43	243	240	214	214
query44	1012	548	573	548
query45	118	104	106	104
query46	787	839	525	525
query47	777	769	777	769
query48	327	322	239	239
query49	560	243	192	192
query50	834	329	251	251
query51	8211	8296	8337	8296
query52	73	78	65	65
query53	199	207	157	157
query54	223	172	171	171
query55	79	60	57	57
query56	231	228	204	204
query57	663	687	653	653
query58	233	192	190	190
query59	1216	1219	1095	1095
query60	258	219	188	188
query61	131	125	120	120
query62	345	201	179	179
query63	184	157	169	157
query64	2878	820	683	683
query65	1644	1619	1740	1619
query66	1917	349	262	262
query67	10387	9959	9862	9862
query68	2875	1159	794	794
query69	356	236	211	211
query70	682	622	649	622
query71	292	286	244	244
query72	2524	1761	1570	1570
query73	644	588	360	360
query74	1848	1255	1169	1169
query75	1245	1170	1017	1017
query76	2287	757	542	542
query77	263	257	214	214
query78	4011	3743	3255	3255
query79	2770	818	623	623
query80	1636	402	340	340
query81	536	203	180	180
query82	681	121	104	104
query83	319	251	234	234
query84	325	123	102	102
query85	885	444	370	370
query86	489	183	174	174
query87	1028	986	901	901
query88	2937	2126	2141	2126
query89	315	223	205	205
query90	2016	146	139	139
query91	158	141	124	124
query92	57	48	47	47
query93	1723	1163	818	818
query94	628	235	196	196
query95	641	345	361	345
query96	802	583	270	270
query97	1041	1062	1040	1040
query98	176	137	130	130
query99	416	352	309	309
Total cold run time: 181011 ms
Total hot run time: 84739 ms

[hello-stephen]

ClickBench: Total hot run time: 14.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2130e12f16144fc369125d34a5dda5309b3e7184, data reload: false

query1	0.01	0.00	0.01
query2	0.08	0.04	0.04
query3	0.24	0.10	0.10
query4	1.61	0.10	0.10
query5	0.17	0.16	0.16
query6	1.25	0.67	0.72
query7	0.03	0.01	0.00
query8	0.04	0.03	0.03
query9	0.29	0.21	0.21
query10	0.36	0.35	0.36
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.31	0.30	0.30
query14	0.46	0.44	0.45
query15	0.36	0.36	0.37
query16	0.23	0.22	0.21
query17	0.71	0.70	0.69
query18	0.18	0.17	0.17
query19	1.24	1.23	1.13
query20	0.02	0.01	0.01
query21	15.45	0.17	0.11
query22	5.06	0.05	0.04
query23	16.16	0.25	0.10
query24	2.99	0.32	0.25
query25	0.10	0.04	0.03
query26	0.74	0.17	0.12
query27	0.03	0.04	0.03
query28	3.61	0.56	0.27
query29	12.43	3.14	2.57
query30	0.26	0.12	0.12
query31	2.76	0.38	0.16
query32	3.50	0.30	0.23
query33	1.36	1.46	1.42
query34	15.41	2.19	1.79
query35	1.75	1.73	1.71
query36	0.46	0.29	0.30
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.11	0.08	0.07
query41	0.08	0.03	0.03
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 90.35 s
Total hot run time: 14.67 s

[hello-stephen]

FE UT Coverage Report

Increment line coverage 77.36% (540/698) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.45% (29116/46620)
Line Coverage	47.48% (304812/641967)
Region Coverage	43.15% (246043/570241)
Branch Coverage	44.68% (114474/256188)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.95% (34292/45153)
Line Coverage	60.89% (386426/634622)
Region Coverage	56.97% (323618/568077)
Branch Coverage	57.81% (147689/255463)

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 68.43% (479/700) 🎉
Increment coverage report
Complete coverage report

[linrrzqqq]

/review

[github-actions]

Automated review status: complete and converged after three rounds. Requesting changes for one new P1 correctness issue.

Finding

• Native-table access-path pruning can lose Map values used only inside a nested Lambda and substitute default/null values; one inline comment is attached.
• The existing P1 variable-width Lambda budget thread remains applicable and was not duplicated.

Critical checkpoint conclusions

• Goal and proof: the PR adds Map Lambda functions, Map constructors/filtering, tuple Lambda syntax, and FE-to-BE lowering. Registrations and ordinary semantics align, and unit/regression sources cover null, empty, duplicate, constant, computed, nondeterministic, nested, aggregate, Join, and Generate cases; the accepted native stored-column boundary is not covered.
• Scope and focus: the implementation is cohesive for the advertised feature and reuses ArrayMap/MapEntries/Map primitives. No additional user focus was supplied.
• Concurrency: execution uses query-local expression/column state; no new shared mutable state, locks, lock ordering, or thread-safety mechanism is introduced.
• Lifecycle/static initialization: Lambda frames use scoped guards and ColumnMap sharing uses COW; no non-intuitive release path, circular ownership, or cross-TU static initialization dependency was found.
• Configuration: no configuration item or dynamic-update path is added.
• Compatibility: FE wrapper names, arities, return/nullability types, and internal physical symbols match BE registration. No storage format, persisted metadata, or protocol variable changes are introduced; the new function symbols are additive.
• Parallel paths: public Map functions, internal unique/filtered constructors, legacy two-argument map_filter, constant folding, computed inputs, nested Lambdas, and native/external scan paths were checked. The native scan divergence is the attached finding.
• Conditional checks: const broadcasting, top-level null Maps/arrays, hidden null-row payloads, mismatched offsets, nullable predicates, and last-win deduplication checks have explicit failure/skip behavior and targeted tests. No separate conditional-check defect survived.
• Test coverage: BE unit tests and FE/regression tests are broad, including negative cases, but need a stored native-table test where the outer Map Lambda retains the key and reads the value only inside an inner Lambda.
• Test results: the committed .out results match the asserted ordinary semantics. Per the review-run contract, I did not run builds or tests.
• Observability: these are scalar analysis/execution paths; existing AnalysisException/Status failures are sufficient, and no new log or metric is required.
• Transactions/persistence: no transaction, EditLog, master-failover, or persistent-state path is changed.
• Data writes/crashes: the feature is query-time only and does not modify stored data. COW detachment and Status propagation are sound outside the accepted read-pruning correctness issue.
• FE-BE variables: no new session/protocol variable is transmitted. FE and BE independently computed types/names were checked and agree.
• Performance: zero-copy/COW paths are reasonable, but the existing live byte-budget P1 remains unresolved; no distinct additional CPU/memory issue survived duplicate fencing.
• Other issues: the scalar-subquery tuple-parser candidate was rejected because the semantic predicate only gates, rather than forces, its alternative. No other unresolved candidate remains.

[linrrzqqq]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 17112 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9651dc40a821a64e8eee1bc21d5fe3ab472bc746, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17560	3053	3010	3010
q2	1874	225	151	151
q3	10485	849	511	511
q4	4671	252	196	196
q5	7679	593	383	383
q6	137	110	99	99
q7	536	516	389	389
q8	9240	922	933	922
q9	3525	2437	2344	2344
q10	6517	898	710	710
q11	434	257	242	242
q12	684	396	328	328
q13	17855	1513	1165	1165
q14	162	148	139	139
q15	q16	431	398	364	364
q17	808	852	797	797
q18	3146	2229	2231	2229
q19	1104	855	783	783
q20	597	501	438	438
q21	5200	1685	1736	1685
q22	323	260	227	227
Total cold run time: 92968 ms
Total hot run time: 17112 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3373	3322	3286	3286
q2	206	208	154	154
q3	2211	2340	2177	2177
q4	1180	1161	887	887
q5	2182	2109	2127	2109
q6	171	122	86	86
q7	1010	905	867	867
q8	1603	1400	1385	1385
q9	3135	3058	3062	3058
q10	1874	1807	1621	1621
q11	350	268	246	246
q12	458	431	332	332
q13	1488	1522	1179	1179
q14	170	164	168	164
q15	q16	391	390	345	345
q17	1040	1032	1022	1022
q18	4887	4343	4860	4343
q19	877	855	842	842
q20	983	929	778	778
q21	3472	3301	3281	3281
q22	410	341	336	336
Total cold run time: 31471 ms
Total hot run time: 28498 ms

[hello-stephen]

TPC-DS: Total hot run time: 82668 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9651dc40a821a64e8eee1bc21d5fe3ab472bc746, data reload: false

query5	4259	397	344	344
query6	398	162	194	162
query7	4840	446	258	258
query8	286	119	115	115
query9	8698	2814	2849	2814
query10	399	244	240	240
query11	5377	1025	908	908
query12	119	76	71	71
query13	1201	449	333	333
query14	6174	2174	2065	2065
query14_1	1971	1945	1913	1913
query15	172	122	110	110
query16	907	376	401	376
query17	808	467	385	385
query18	2330	337	240	240
query19	167	142	113	113
query20	71	68	78	68
query21	215	117	103	103
query22	5277	5320	5170	5170
query23	6752	6310	6119	6119
query23_1	5929	6109	6006	6006
query24	7269	1112	771	771
query24_1	764	773	773	773
query25	407	288	245	245
query26	1255	264	156	156
query27	2725	443	285	285
query28	4637	1482	1486	1482
query29	915	432	344	344
query30	265	179	154	154
query31	834	418	351	351
query32	96	48	48	48
query33	451	213	162	162
query34	1010	854	484	484
query35	404	400	331	331
query36	553	536	540	536
query37	117	78	67	67
query38	993	836	825	825
query39	543	468	471	468
query39_1	477	481	510	481
query40	216	121	113	113
query41	52	50	50	50
query42	78	75	81	75
query43	235	235	208	208
query44	1008	541	565	541
query45	116	105	100	100
query46	739	851	507	507
query47	765	733	696	696
query48	318	294	228	228
query49	539	231	180	180
query50	807	324	264	264
query51	7902	7870	7931	7870
query52	78	85	71	71
query53	199	198	163	163
query54	265	175	171	171
query55	73	62	57	57
query56	273	219	222	219
query57	673	651	646	646
query58	228	209	192	192
query59	1196	1200	1111	1111
query60	254	206	216	206
query61	136	138	131	131
query62	340	212	174	174
query63	185	153	161	153
query64	2936	785	657	657
query65	1629	1572	1558	1558
query66	1979	314	248	248
query67	10288	9971	9563	9563
query68	3033	1202	718	718
query69	332	210	191	191
query70	670	583	607	583
query71	314	249	231	231
query72	2345	1750	1554	1554
query73	642	641	350	350
query74	2006	1208	1135	1135
query75	1243	1142	1013	1013
query76	2388	733	544	544
query77	262	265	208	208
query78	3844	3608	3188	3188
query79	2114	909	615	615
query80	1597	388	343	343
query81	487	195	182	182
query82	624	128	99	99
query83	322	248	231	231
query84	307	121	105	105
query85	844	438	396	396
query86	391	177	168	168
query87	991	966	898	898
query88	2778	2129	2118	2118
query89	302	221	204	204
query90	2024	146	149	146
query91	157	151	119	119
query92	52	48	44	44
query93	1503	1160	762	762
query94	634	246	224	224
query95	637	360	396	360
query96	776	561	253	253
query97	1075	1034	1011	1011
query98	145	133	132	132
query99	411	353	310	310
Total cold run time: 178242 ms
Total hot run time: 82668 ms

[hello-stephen]

ClickBench: Total hot run time: 14.61 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9651dc40a821a64e8eee1bc21d5fe3ab472bc746, data reload: false

query1	0.01	0.00	0.00
query2	0.07	0.03	0.03
query3	0.24	0.11	0.09
query4	1.61	0.11	0.10
query5	0.18	0.16	0.15
query6	1.26	0.67	0.67
query7	0.03	0.01	0.00
query8	0.05	0.03	0.04
query9	0.29	0.21	0.21
query10	0.35	0.34	0.36
query11	0.17	0.11	0.12
query12	0.15	0.12	0.11
query13	0.30	0.31	0.31
query14	0.44	0.45	0.46
query15	0.36	0.36	0.33
query16	0.21	0.20	0.22
query17	0.69	0.72	0.74
query18	0.17	0.16	0.17
query19	1.19	1.19	1.22
query20	0.02	0.01	0.01
query21	15.45	0.16	0.12
query22	5.10	0.04	0.05
query23	16.18	0.26	0.10
query24	2.97	0.32	0.26
query25	0.12	0.03	0.03
query26	0.79	0.16	0.13
query27	0.04	0.03	0.03
query28	3.63	0.55	0.28
query29	12.44	3.21	2.54
query30	0.27	0.11	0.12
query31	2.76	0.38	0.17
query32	3.54	0.33	0.23
query33	1.49	1.44	1.38
query34	15.40	2.19	1.75
query35	1.74	1.76	1.71
query36	0.45	0.30	0.27
query37	0.06	0.04	0.04
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.10	0.08	0.07
query41	0.08	0.03	0.02
query42	0.03	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 90.55 s
Total hot run time: 14.61 s

[hello-stephen]

FE UT Coverage Report

Increment line coverage 82.27% (283/344) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.47% (29128/46629)
Line Coverage	47.49% (304953/642120)
Region Coverage	43.12% (245929/570358)
Branch Coverage	44.69% (114516/256256)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.93% (34292/45162)
Line Coverage	60.91% (386640/634775)
Region Coverage	57.05% (324146/568194)
Branch Coverage	57.85% (147817/255531)

[hello-stephen]

TPC-H: Total hot run time: 16923 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6e16d36007e68b5d891d8924ca90d92695f829b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17574	3097	3062	3062
q2	2069	259	220	220
q3	10254	851	522	522
q4	4674	249	205	205
q5	7670	557	383	383
q6	139	112	97	97
q7	523	501	398	398
q8	9254	904	910	904
q9	3486	2395	2397	2395
q10	6517	851	688	688
q11	402	197	182	182
q12	626	270	203	203
q13	18098	1546	1162	1162
q14	159	150	143	143
q15	q16	429	397	372	372
q17	1434	894	796	796
q18	3072	2243	2248	2243
q19	1133	895	807	807
q20	382	278	204	204
q21	5209	1712	1878	1712
q22	329	270	225	225
Total cold run time: 93433 ms
Total hot run time: 16923 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3433	3370	3332	3332
q2	519	393	372	372
q3	2190	2354	2138	2138
q4	1190	1169	889	889
q5	2176	2115	2097	2097
q6	165	117	84	84
q7	1047	925	843	843
q8	1616	1418	1416	1416
q9	3147	3120	3102	3102
q10	1862	1799	1617	1617
q11	354	269	255	255
q12	448	426	343	343
q13	1483	1555	1170	1170
q14	177	164	153	153
q15	q16	399	397	354	354
q17	3592	3295	3177	3177
q18	4850	4483	4772	4483
q19	3507	890	847	847
q20	995	939	806	806
q21	3798	3061	3185	3061
q22	391	341	326	326
Total cold run time: 37339 ms
Total hot run time: 30865 ms

[hello-stephen]

FE UT Coverage Report

Increment line coverage 80.38% (213/265) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

TPC-DS: Total hot run time: 81934 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6e16d36007e68b5d891d8924ca90d92695f829b, data reload: false

query5	4286	409	338	338
query6	392	136	122	122
query7	4955	416	237	237
query8	285	123	117	117
query9	8682	2898	2905	2898
query10	408	217	176	176
query11	5388	1029	911	911
query12	113	69	72	69
query13	1228	440	301	301
query14	6061	2175	2074	2074
query14_1	1951	1964	1949	1949
query15	178	122	104	104
query16	898	366	275	275
query17	785	427	371	371
query18	2325	325	227	227
query19	166	135	99	99
query20	72	68	67	67
query21	199	100	86	86
query22	5462	5386	5303	5303
query23	6376	6256	6118	6118
query23_1	5946	6099	5968	5968
query24	7304	1063	733	733
query24_1	774	763	776	763
query25	405	272	263	263
query26	1089	238	124	124
query27	2771	404	246	246
query28	4698	1489	1506	1489
query29	940	430	331	331
query30	251	152	128	128
query31	826	386	313	313
query32	122	71	68	68
query33	455	205	165	165
query34	991	788	480	480
query35	413	411	337	337
query36	561	574	546	546
query37	116	83	70	70
query38	994	832	830	830
query39	510	475	466	466
query39_1	454	454	450	450
query40	201	87	79	79
query41	56	50	51	50
query42	79	73	73	73
query43	239	245	211	211
query44	1011	538	542	538
query45	107	103	102	102
query46	800	862	513	513
query47	745	763	720	720
query48	299	294	228	228
query49	528	244	186	186
query50	766	267	198	198
query51	8258	8246	8266	8246
query52	74	68	63	63
query53	198	197	144	144
query54	234	175	160	160
query55	73	59	55	55
query56	198	175	189	175
query57	696	698	665	665
query58	199	160	169	160
query59	1238	1217	1090	1090
query60	242	192	182	182
query61	134	143	134	134
query62	403	213	175	175
query63	179	141	141	141
query64	2656	708	599	599
query65	1613	1595	1627	1595
query66	1844	253	205	205
query67	9976	9661	9564	9564
query68	3037	1131	756	756
query69	336	218	202	202
query70	686	625	618	618
query71	245	166	165	165
query72	2362	1728	1593	1593
query73	656	650	328	328
query74	1978	1217	1140	1140
query75	1170	1101	935	935
query76	2350	732	526	526
query77	245	258	210	210
query78	3925	3673	3153	3153
query79	2856	824	560	560
query80	1574	323	276	276
query81	515	154	135	135
query82	635	132	92	92
query83	290	213	202	202
query84	295	111	89	89
query85	818	348	304	304
query86	404	175	177	175
query87	1006	963	897	897
query88	2962	2128	2077	2077
query89	288	200	177	177
query90	1947	129	133	129
query91	130	120	103	103
query92	81	69	71	69
query93	1878	1084	725	725
query94	650	237	227	227
query95	529	323	227	227
query96	794	591	283	283
query97	1088	1067	1039	1039
query98	162	130	130	130
query99	414	347	306	306
Total cold run time: 178610 ms
Total hot run time: 81934 ms

[hello-stephen]

ClickBench: Total hot run time: 14.62 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6e16d36007e68b5d891d8924ca90d92695f829b, data reload: false

query1	0.00	0.01	0.00
query2	0.07	0.03	0.04
query3	0.25	0.10	0.11
query4	1.60	0.10	0.10
query5	0.17	0.15	0.16
query6	1.25	0.69	0.70
query7	0.04	0.01	0.00
query8	0.04	0.03	0.03
query9	0.29	0.22	0.21
query10	0.34	0.35	0.34
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.32	0.31	0.32
query14	0.45	0.45	0.45
query15	0.37	0.33	0.34
query16	0.22	0.22	0.22
query17	0.66	0.73	0.69
query18	0.17	0.16	0.18
query19	1.21	1.13	1.17
query20	0.02	0.01	0.01
query21	15.43	0.16	0.12
query22	5.06	0.04	0.04
query23	16.18	0.24	0.11
query24	3.01	0.32	0.25
query25	0.10	0.04	0.04
query26	0.83	0.16	0.12
query27	0.04	0.03	0.03
query28	3.68	0.55	0.26
query29	12.46	3.14	2.53
query30	0.25	0.12	0.12
query31	2.75	0.37	0.16
query32	3.53	0.32	0.23
query33	1.36	1.51	1.47
query34	15.37	2.17	1.77
query35	1.74	1.73	1.68
query36	0.46	0.29	0.28
query37	0.06	0.04	0.04
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.10	0.08	0.07
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.42 s
Total hot run time: 14.62 s

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.05% (34412/45247)
Line Coverage	61.00% (387754/635646)
Region Coverage	57.02% (324574/569250)
Branch Coverage	57.96% (148387/256027)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.05% (34412/45247)
Line Coverage	61.00% (387754/635646)
Region Coverage	57.02% (324574/569250)
Branch Coverage	57.96% (148387/256027)

[morrySnow]

这个 {isTupleLambdaBody()}? 做过 benchmark 吗？确定可以提升性能吗？

[linrrzqqq]

只是为了支持语法糖 (k, v) -> (k * 3, v + 1), 估计还可能存在回退

[morrySnow]

没有这个也可以支持的吧，这里只不过在不满足的情况下，去掉了这个分支

[morrySnow]

这个可能能提升parser的性能，需要bench一下看看，bench可以参考：#66902

[linrrzqqq]

原先是不支持这个语法糖的

Doris> select array_map((k, v) -> (k + 1, v + 1), [1], [1]);
、ERROR 1105 (HY000): errCode = 2, detailMessage = 
missing ')' at ','(line 1, pos 33)

Doris> select array_map((k, v) -> struct(k + 1, v + 1), [1], [1]);
+-----------------------------------------------------+
| array_map((k, v) -> struct(k + 1, v + 1), [1], [1]) |
+-----------------------------------------------------+
| [{"col1":2, "col2":2}]                              |
+-----------------------------------------------------+

测了下，性能稳定劣化， 可能没支持的必要?

SQL 场景	无 predicate	当前 predicate	变化
单个 tuple lambda	23.749 μs/op	25.153 μs/op	慢 5.91%
32 个 tuple lambda	679.133 μs/op	697.665 μs/op	慢 2.73%
单个括号普通表达式	20.621 μs/op	20.734 μs/op	慢 0.55%
32 个括号普通表达式	572.181 μs/op	575.088 μs/op	慢 0.51%
32 个普通表达式	336.354 μs/op	338.058 μs/op	慢 0.51%

[morrySnow]

这个list，包括下面的 bindMapLambdaFunction 可以重构以更适合扩展，参考：#67284

[linrrzqqq]

run buildall

[hello-stephen]

FE UT Coverage Report

Increment line coverage 80.11% (290/362) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.76% (29386/46823)
Line Coverage	47.75% (307452/643944)
Region Coverage	43.36% (248302/572604)
Branch Coverage	44.93% (115593/257272)

[hello-stephen]

TPC-H: Total hot run time: 16843 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 21af11aee56d1953fd173513fb4b9ae8f56767a9, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17574	3017	2986	2986
q2	2080	274	227	227
q3	10226	859	531	531
q4	4668	253	201	201
q5	7674	574	392	392
q6	135	116	98	98
q7	541	514	382	382
q8	9259	883	964	883
q9	3485	2406	2390	2390
q10	6512	847	708	708
q11	389	199	182	182
q12	605	266	200	200
q13	18147	1502	1158	1158
q14	159	151	146	146
q15	q16	429	394	370	370
q17	1296	890	856	856
q18	3033	2250	2246	2246
q19	1125	885	762	762
q20	356	289	203	203
q21	4819	1693	1846	1693
q22	324	270	229	229
Total cold run time: 92836 ms
Total hot run time: 16843 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3330	3288	3282	3282
q2	493	391	375	375
q3	2239	2281	2165	2165
q4	1178	1164	884	884
q5	2179	2092	2104	2092
q6	170	118	86	86
q7	1043	928	909	909
q8	1593	1406	1412	1406
q9	3107	3093	3069	3069
q10	1868	1801	1630	1630
q11	350	268	250	250
q12	442	429	333	333
q13	1448	1553	1161	1161
q14	169	168	154	154
q15	q16	396	393	361	361
q17	3675	3342	3170	3170
q18	4814	4420	4683	4420
q19	838	804	794	794
q20	1129	996	844	844
q21	3815	3119	3287	3119
q22	405	345	312	312
Total cold run time: 34681 ms
Total hot run time: 30816 ms

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.21% (34560/45351)
Line Coverage	61.21% (389677/636587)
Region Coverage	57.42% (327544/570422)
Branch Coverage	58.24% (149417/256539)

[hello-stephen]

TPC-DS: Total hot run time: 81668 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 21af11aee56d1953fd173513fb4b9ae8f56767a9, data reload: false

query5	4289	408	347	347
query6	378	141	121	121
query7	4957	376	237	237
query8	296	125	115	115
query9	8680	2859	2869	2859
query10	391	221	191	191
query11	5387	1020	928	928
query12	116	71	67	67
query13	1185	439	318	318
query14	6050	2189	2070	2070
query14_1	1956	1960	1956	1956
query15	170	117	107	107
query16	902	363	341	341
query17	808	423	351	351
query18	2312	307	231	231
query19	171	133	109	109
query20	68	67	68	67
query21	192	100	85	85
query22	5438	5486	5282	5282
query23	6627	6305	5956	5956
query23_1	6056	6118	6098	6098
query24	7246	1078	751	751
query24_1	773	781	783	781
query25	414	287	240	240
query26	1223	229	148	148
query27	2773	428	249	249
query28	4689	1500	1497	1497
query29	910	435	342	342
query30	244	150	128	128
query31	810	400	323	323
query32	126	74	70	70
query33	457	213	172	172
query34	993	819	467	467
query35	405	395	329	329
query36	579	555	526	526
query37	116	78	69	69
query38	990	840	807	807
query39	486	484	465	465
query39_1	444	450	486	450
query40	202	92	73	73
query41	53	52	53	52
query42	78	78	77	77
query43	237	241	208	208
query44	1020	552	546	546
query45	110	108	97	97
query46	752	807	543	543
query47	756	759	694	694
query48	297	315	234	234
query49	529	224	192	192
query50	749	265	194	194
query51	7968	8042	8005	8005
query52	88	67	61	61
query53	206	202	146	146
query54	246	200	165	165
query55	81	60	56	56
query56	214	192	171	171
query57	705	651	661	651
query58	219	187	174	174
query59	1301	1261	1083	1083
query60	243	195	176	176
query61	134	129	128	128
query62	357	201	185	185
query63	174	144	151	144
query64	2837	748	599	599
query65	1621	1566	1609	1566
query66	1932	267	215	215
query67	9761	9865	9522	9522
query68	2912	1214	728	728
query69	337	213	199	199
query70	675	619	614	614
query71	258	184	164	164
query72	2328	1744	1584	1584
query73	620	583	340	340
query74	1970	1229	1140	1140
query75	1182	1098	943	943
query76	2312	732	556	556
query77	258	262	208	208
query78	4059	3676	3201	3201
query79	2789	859	577	577
query80	1577	324	299	299
query81	513	154	131	131
query82	619	122	100	100
query83	284	212	197	197
query84	294	110	89	89
query85	815	352	296	296
query86	472	177	168	168
query87	1028	970	876	876
query88	3151	2109	2109	2109
query89	286	198	173	173
query90	2170	126	125	125
query91	133	123	101	101
query92	103	72	71	71
query93	2432	1115	736	736
query94	624	240	213	213
query95	509	324	228	228
query96	803	564	271	271
query97	1042	1036	989	989
query98	176	141	129	129
query99	416	347	308	308
Total cold run time: 179532 ms
Total hot run time: 81668 ms

[hello-stephen]

ClickBench: Total hot run time: 14.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 21af11aee56d1953fd173513fb4b9ae8f56767a9, data reload: false

query1	0.00	0.01	0.00
query2	0.08	0.04	0.04
query3	0.24	0.11	0.11
query4	1.60	0.10	0.10
query5	0.18	0.15	0.16
query6	1.24	0.72	0.66
query7	0.03	0.00	0.00
query8	0.04	0.04	0.04
query9	0.29	0.22	0.21
query10	0.34	0.34	0.34
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.30	0.31	0.31
query14	0.47	0.45	0.45
query15	0.37	0.34	0.33
query16	0.22	0.23	0.22
query17	0.67	0.70	0.71
query18	0.18	0.15	0.16
query19	1.19	1.15	1.21
query20	0.01	0.01	0.01
query21	15.44	0.16	0.11
query22	5.08	0.05	0.04
query23	16.16	0.25	0.10
query24	2.97	0.33	0.26
query25	0.11	0.04	0.04
query26	0.74	0.16	0.12
query27	0.05	0.03	0.02
query28	3.65	0.55	0.29
query29	12.43	3.15	2.54
query30	0.26	0.11	0.11
query31	2.76	0.36	0.17
query32	3.53	0.33	0.22
query33	1.36	1.40	1.43
query34	15.35	2.14	1.78
query35	1.74	1.74	1.75
query36	0.46	0.31	0.28
query37	0.06	0.04	0.04
query38	0.05	0.03	0.03
query39	0.03	0.02	0.03
query40	0.12	0.08	0.07
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 90.27 s
Total hot run time: 14.64 s

[morrySnow]

把所有lambdaScalar 放到一起吧，按字母序排序。

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 67.12% (298/444) 🎉
Increment coverage report
Complete coverage report

[linrrzqqq]

run buildall

[hello-stephen]

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.78% (29397/46825)
Line Coverage	47.79% (307755/643968)
Region Coverage	43.35% (248250/572616)
Branch Coverage	44.94% (115615/257282)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.03% (34479/45351)
Line Coverage	61.02% (388451/636604)
Region Coverage	57.10% (325733/570430)
Branch Coverage	57.96% (148702/256545)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.86% (34405/45351)
Line Coverage	60.86% (387429/636604)
Region Coverage	56.93% (324727/570430)
Branch Coverage	57.80% (148286/256545)

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 41.22% (298/723) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.86% (34405/45351)
Line Coverage	60.86% (387423/636604)
Region Coverage	56.92% (324669/570430)
Branch Coverage	57.80% (148294/256545)

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 40.47% (295/729) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.87% (34406/45351)
Line Coverage	60.86% (387425/636604)
Region Coverage	56.92% (324686/570430)
Branch Coverage	57.81% (148299/256545)