Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement](paimon)support paimon top level schema change. #48723

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

hubgeter
Copy link
Contributor

@hubgeter hubgeter commented Mar 6, 2025

What problem does this PR solve?

Problem Summary:
Supports reading tables after the top-level schema of paimon is changed, but does not support tables after the internal schema of struct is changed.

change top-level schema(support):

--spark sql 
ALTER TABLE table_name ADD COLUMNS (c1 INT,c2 STRING);
ALTER TABLE table_name RENAME COLUMN c0 TO c1;
ALTER TABLE table_name DROP COLUMNS (c1, c2);
ALTER TABLE table_name ADD COLUMN c INT FIRST;
ALTER TABLE table_name ADD COLUMN c INT AFTER b;
ALTER TABLE table_name ALTER COLUMN col_a FIRST;
ALTER TABLE table_name ALTER COLUMN col_a AFTER col_b;

change internal schema of struct schema(not support, will support in the next PR):

--spark sql 
ALTER TABLE table_name ADD COLUMN        v.value.f3 STRING;
ALTER TABLE table_name RENAME COLUMN v.f1 to f100;
ALTER TABLE table_name DROP COLUMN      v.value.f3 ;
ALTER TABLE table_name ALTER COLUMN      v.col_a FIRST;

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hubgeter hubgeter marked this pull request as draft March 6, 2025 02:42
@hubgeter hubgeter closed this Mar 7, 2025
@hubgeter hubgeter reopened this Mar 7, 2025
@hubgeter
Copy link
Contributor Author

hubgeter commented Mar 7, 2025

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.70% (17634/26840)
Region Coverage: 65.14% (8690/13340)
Branch Coverage: 55.12% (4691/8510)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a5f09cd2ab08a54ba551df7fffafe59a39ad3a95_a5f09cd2ab08a54ba551df7fffafe59a39ad3a95_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32681 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a5f09cd2ab08a54ba551df7fffafe59a39ad3a95, data reload: false

------ Round 1 ----------------------------------
q1	17616	5244	5053	5053
q2	2052	306	168	168
q3	10384	1287	747	747
q4	10217	1052	522	522
q5	7516	2479	2378	2378
q6	198	172	137	137
q7	922	749	599	599
q8	9304	1316	1166	1166
q9	5031	4878	4681	4681
q10	6837	2322	1872	1872
q11	483	283	267	267
q12	346	348	222	222
q13	17773	3701	3065	3065
q14	227	221	214	214
q15	543	495	487	487
q16	646	611	586	586
q17	594	870	333	333
q18	7020	6494	6504	6494
q19	2031	953	554	554
q20	326	320	192	192
q21	2797	2171	1957	1957
q22	1093	1044	987	987
Total cold run time: 103956 ms
Total hot run time: 32681 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5250	5167	5162	5162
q2	240	332	234	234
q3	2194	2695	2316	2316
q4	1445	1924	1375	1375
q5	4265	4211	4181	4181
q6	214	163	126	126
q7	2005	1926	1787	1787
q8	2646	2634	2628	2628
q9	7344	7288	7161	7161
q10	3093	3196	2805	2805
q11	584	511	488	488
q12	678	768	574	574
q13	3499	3890	3250	3250
q14	287	302	268	268
q15	607	507	510	507
q16	694	687	646	646
q17	1167	1574	1379	1379
q18	7893	7793	7583	7583
q19	869	818	841	818
q20	2001	2036	1873	1873
q21	5483	4903	4727	4727
q22	1126	1057	1002	1002
Total cold run time: 53584 ms
Total hot run time: 50890 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192362 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a5f09cd2ab08a54ba551df7fffafe59a39ad3a95, data reload: false

query1	1355	1006	999	999
query2	6213	1890	1917	1890
query3	11092	4648	4386	4386
query4	54056	26431	23441	23441
query5	5232	522	493	493
query6	342	187	183	183
query7	4925	497	283	283
query8	307	233	238	233
query9	5851	2529	2563	2529
query10	427	313	264	264
query11	15216	15069	14924	14924
query12	156	110	112	110
query13	1099	520	400	400
query14	10146	6484	7039	6484
query15	199	191	174	174
query16	7139	684	474	474
query17	1073	701	577	577
query18	1548	406	309	309
query19	193	208	166	166
query20	121	127	127	127
query21	211	127	110	110
query22	4668	4747	4623	4623
query23	33902	33198	33240	33198
query24	5658	2403	2419	2403
query25	459	464	395	395
query26	675	279	155	155
query27	1660	485	331	331
query28	2829	2441	2441	2441
query29	580	565	428	428
query30	279	218	199	199
query31	867	881	812	812
query32	77	72	62	62
query33	461	382	326	326
query34	764	863	492	492
query35	829	842	773	773
query36	971	990	907	907
query37	124	105	85	85
query38	4157	4370	4261	4261
query39	1514	1440	1443	1440
query40	218	157	105	105
query41	51	54	49	49
query42	128	102	101	101
query43	498	514	486	486
query44	1314	831	814	814
query45	181	175	172	172
query46	857	1061	652	652
query47	1868	1907	1769	1769
query48	388	426	320	320
query49	715	530	445	445
query50	701	752	435	435
query51	4311	4302	4257	4257
query52	104	109	98	98
query53	237	255	191	191
query54	492	518	412	412
query55	92	91	79	79
query56	269	276	243	243
query57	1194	1181	1113	1113
query58	295	230	243	230
query59	2795	2964	2812	2812
query60	281	278	264	264
query61	123	117	123	117
query62	751	732	677	677
query63	231	202	195	195
query64	1476	1038	679	679
query65	4604	4536	4369	4369
query66	733	388	300	300
query67	15855	15514	15514	15514
query68	7247	881	507	507
query69	534	304	262	262
query70	1138	1059	1114	1059
query71	447	287	265	265
query72	5757	3592	3704	3592
query73	1303	736	348	348
query74	8981	9016	8892	8892
query75	3229	3204	2720	2720
query76	3859	1193	760	760
query77	531	351	289	289
query78	9994	10112	9261	9261
query79	2261	828	597	597
query80	673	579	450	450
query81	483	260	226	226
query82	451	125	95	95
query83	175	168	151	151
query84	285	92	77	77
query85	766	352	297	297
query86	399	320	272	272
query87	4447	4456	4439	4439
query88	3415	2214	2219	2214
query89	398	308	276	276
query90	1812	208	211	208
query91	160	140	107	107
query92	75	59	57	57
query93	2152	1048	583	583
query94	696	409	310	310
query95	347	262	250	250
query96	480	565	275	275
query97	3372	3425	3294	3294
query98	228	203	201	201
query99	1368	1368	1243	1243
Total cold run time: 295600 ms
Total hot run time: 192362 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a5f09cd2ab08a54ba551df7fffafe59a39ad3a95, data reload: false

query1	0.03	0.03	0.03
query2	0.10	0.04	0.05
query3	0.27	0.05	0.05
query4	1.61	0.08	0.08
query5	0.55	0.55	0.55
query6	1.20	0.72	0.73
query7	0.02	0.02	0.02
query8	0.06	0.05	0.05
query9	0.62	0.52	0.51
query10	0.57	0.60	0.58
query11	0.25	0.13	0.12
query12	0.25	0.13	0.12
query13	0.64	0.62	0.62
query14	2.85	2.80	2.81
query15	1.01	0.88	0.89
query16	0.38	0.37	0.36
query17	1.03	1.04	1.03
query18	0.18	0.19	0.18
query19	1.96	1.93	1.88
query20	0.02	0.01	0.02
query21	15.34	0.97	0.66
query22	0.92	1.02	0.76
query23	14.71	1.49	0.74
query24	5.46	0.57	0.28
query25	0.17	0.09	0.09
query26	0.55	0.23	0.18
query27	0.08	0.09	0.08
query28	10.95	1.18	0.56
query29	12.53	4.08	3.38
query30	0.27	0.08	0.06
query31	2.82	0.60	0.43
query32	3.22	0.59	0.50
query33	3.00	3.05	3.08
query34	16.64	5.05	4.39
query35	4.46	4.47	4.48
query36	0.62	0.49	0.49
query37	0.21	0.17	0.17
query38	0.17	0.15	0.15
query39	0.05	0.04	0.05
query40	0.19	0.16	0.15
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.05
Total cold run time: 106.17 s
Total hot run time: 31.4 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/302) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 46.84% (12531/26750)
Line Coverage 36.43% (106831/293289)
Region Coverage 35.46% (54554/153834)
Branch Coverage 30.82% (27437/89026)

@hubgeter
Copy link
Contributor Author

hubgeter commented Mar 8, 2025

run buildall

@hubgeter
Copy link
Contributor Author

hubgeter commented Mar 8, 2025

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.69% (17632/26840)
Region Coverage: 65.16% (8692/13340)
Branch Coverage: 55.12% (4691/8510)
Coverage Report: http://coverage.selectdb-in.cc/coverage/10a89015177ef906757e78bb85e91b51a1914ca6_10a89015177ef906757e78bb85e91b51a1914ca6_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32633 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10a89015177ef906757e78bb85e91b51a1914ca6, data reload: false

------ Round 1 ----------------------------------
q1	17629	5247	5132	5132
q2	2054	308	174	174
q3	10728	1236	768	768
q4	10372	1025	518	518
q5	9077	2470	2299	2299
q6	197	169	130	130
q7	895	744	618	618
q8	9297	1344	1165	1165
q9	4986	4855	4643	4643
q10	6916	2315	1881	1881
q11	468	284	264	264
q12	350	353	221	221
q13	17772	3671	3067	3067
q14	239	222	212	212
q15	532	491	491	491
q16	626	615	580	580
q17	588	866	345	345
q18	6725	6484	6451	6451
q19	2087	935	552	552
q20	307	317	192	192
q21	2923	2147	1951	1951
q22	1040	1030	979	979
Total cold run time: 105808 ms
Total hot run time: 32633 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5267	5100	5110	5100
q2	240	329	239	239
q3	2165	2711	2278	2278
q4	1430	1874	1368	1368
q5	4265	4161	4120	4120
q6	226	172	128	128
q7	1943	1917	1776	1776
q8	2605	2592	2636	2592
q9	7239	7157	7201	7157
q10	3031	3209	2798	2798
q11	559	493	489	489
q12	674	721	580	580
q13	3545	3937	3241	3241
q14	291	302	272	272
q15	521	477	477	477
q16	635	703	653	653
q17	1125	1649	1303	1303
q18	7753	7609	7498	7498
q19	843	822	849	822
q20	2042	2037	1849	1849
q21	5377	4977	4761	4761
q22	1092	1066	996	996
Total cold run time: 52868 ms
Total hot run time: 50497 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192050 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10a89015177ef906757e78bb85e91b51a1914ca6, data reload: false

query1	1383	1019	1018	1018
query2	6137	1893	1939	1893
query3	11044	4649	4385	4385
query4	53869	24907	23546	23546
query5	5002	587	485	485
query6	350	225	196	196
query7	4874	505	294	294
query8	309	253	238	238
query9	5419	2535	2532	2532
query10	424	318	261	261
query11	15197	15117	14883	14883
query12	163	112	108	108
query13	1038	507	425	425
query14	10189	6273	6464	6273
query15	211	212	192	192
query16	7112	698	463	463
query17	1043	719	555	555
query18	1587	419	318	318
query19	206	188	168	168
query20	143	123	118	118
query21	207	120	103	103
query22	4611	4371	4410	4371
query23	34280	33340	33589	33340
query24	6527	2438	2392	2392
query25	440	469	402	402
query26	685	285	160	160
query27	2117	512	330	330
query28	3095	2448	2390	2390
query29	577	558	417	417
query30	280	223	193	193
query31	902	916	790	790
query32	74	65	62	62
query33	460	388	316	316
query34	767	872	499	499
query35	825	841	765	765
query36	989	979	893	893
query37	113	105	76	76
query38	4302	4192	4248	4192
query39	1495	1448	1461	1448
query40	213	124	108	108
query41	61	59	63	59
query42	120	102	105	102
query43	511	536	489	489
query44	1325	810	795	795
query45	180	176	170	170
query46	866	1068	690	690
query47	1820	1899	1826	1826
query48	384	414	304	304
query49	705	512	445	445
query50	701	735	422	422
query51	4329	4318	4302	4302
query52	105	104	99	99
query53	237	254	186	186
query54	491	493	412	412
query55	85	79	81	79
query56	298	258	255	255
query57	1186	1203	1132	1132
query58	246	241	241	241
query59	2783	2842	2720	2720
query60	288	278	258	258
query61	119	115	118	115
query62	740	716	683	683
query63	229	195	193	193
query64	1467	1053	714	714
query65	4510	4338	4340	4338
query66	722	389	288	288
query67	15911	15527	15520	15520
query68	6490	908	520	520
query69	542	314	276	276
query70	1217	1141	1105	1105
query71	449	285	258	258
query72	5787	3643	3793	3643
query73	1332	738	345	345
query74	9024	9220	8832	8832
query75	3215	3180	2707	2707
query76	3811	1181	755	755
query77	554	367	272	272
query78	10065	10133	9381	9381
query79	1960	830	580	580
query80	614	515	425	425
query81	487	260	224	224
query82	264	127	99	99
query83	181	179	152	152
query84	281	99	72	72
query85	769	347	312	312
query86	418	300	282	282
query87	4422	4450	4352	4352
query88	3644	2179	2202	2179
query89	406	307	284	284
query90	1766	206	203	203
query91	138	135	109	109
query92	75	61	58	58
query93	1440	1047	587	587
query94	684	412	307	307
query95	345	279	261	261
query96	491	555	275	275
query97	3366	3421	3289	3289
query98	224	202	235	202
query99	1343	1401	1269	1269
Total cold run time: 294882 ms
Total hot run time: 192050 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10a89015177ef906757e78bb85e91b51a1914ca6, data reload: false

query1	0.04	0.04	0.03
query2	0.11	0.04	0.05
query3	0.27	0.05	0.05
query4	1.61	0.07	0.07
query5	0.55	0.55	0.53
query6	1.19	0.72	0.72
query7	0.02	0.01	0.03
query8	0.05	0.04	0.04
query9	0.62	0.53	0.52
query10	0.58	0.60	0.58
query11	0.25	0.12	0.12
query12	0.25	0.13	0.13
query13	0.64	0.62	0.62
query14	2.66	2.69	2.84
query15	1.01	0.89	0.87
query16	0.37	0.36	0.36
query17	1.03	1.04	1.05
query18	0.18	0.19	0.18
query19	2.02	1.85	1.83
query20	0.02	0.02	0.02
query21	15.35	0.97	0.66
query22	0.94	1.06	0.78
query23	14.73	1.51	0.76
query24	5.52	0.56	0.29
query25	0.17	0.09	0.09
query26	0.56	0.21	0.19
query27	0.08	0.08	0.09
query28	10.98	1.15	0.58
query29	12.52	4.15	3.37
query30	0.28	0.08	0.06
query31	2.81	0.64	0.44
query32	3.24	0.60	0.50
query33	3.08	3.04	3.06
query34	16.64	5.14	4.47
query35	4.50	4.47	4.45
query36	0.63	0.50	0.49
query37	0.21	0.17	0.17
query38	0.16	0.16	0.15
query39	0.05	0.04	0.04
query40	0.19	0.16	0.17
query41	0.10	0.05	0.05
query42	0.07	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 106.33 s
Total hot run time: 31.36 s

@hubgeter
Copy link
Contributor Author

hubgeter commented Mar 9, 2025

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.68% (17628/26840)
Region Coverage: 65.13% (8688/13340)
Branch Coverage: 55.08% (4687/8510)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6a516d1ed9fe329c934eeeeb485136333c5935dc_6a516d1ed9fe329c934eeeeb485136333c5935dc_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32645 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a516d1ed9fe329c934eeeeb485136333c5935dc, data reload: false

------ Round 1 ----------------------------------
q1	17686	5182	5105	5105
q2	2050	306	181	181
q3	10741	1315	728	728
q4	10312	1050	528	528
q5	9211	2417	2406	2406
q6	229	182	135	135
q7	918	753	587	587
q8	9312	1260	1150	1150
q9	5572	4757	4707	4707
q10	6828	2317	1892	1892
q11	487	268	270	268
q12	346	366	217	217
q13	17762	3675	3041	3041
q14	233	231	201	201
q15	547	512	502	502
q16	614	625	564	564
q17	586	875	347	347
q18	7154	6416	6452	6416
q19	1302	959	550	550
q20	328	336	191	191
q21	2817	2231	1955	1955
q22	1065	994	974	974
Total cold run time: 106100 ms
Total hot run time: 32645 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5236	5141	5154	5141
q2	238	334	229	229
q3	2153	2721	2308	2308
q4	1453	1873	1406	1406
q5	4257	4118	4202	4118
q6	234	175	126	126
q7	2041	1941	1825	1825
q8	2635	2626	2584	2584
q9	7284	7177	7216	7177
q10	3021	3215	2802	2802
q11	580	522	500	500
q12	669	760	631	631
q13	3524	3906	3206	3206
q14	275	287	284	284
q15	570	501	507	501
q16	645	704	625	625
q17	1158	1620	1309	1309
q18	7670	7797	7490	7490
q19	824	826	999	826
q20	1961	2042	1905	1905
q21	5399	4900	4798	4798
q22	1111	1037	984	984
Total cold run time: 52938 ms
Total hot run time: 50775 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184942 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a516d1ed9fe329c934eeeeb485136333c5935dc, data reload: false

query1	962	398	374	374
query2	6519	1935	1943	1935
query3	6814	217	214	214
query4	26323	23740	22872	22872
query5	4347	674	505	505
query6	307	198	197	197
query7	4608	511	288	288
query8	304	243	237	237
query9	8631	2504	2508	2504
query10	474	307	245	245
query11	15937	15086	14993	14993
query12	162	106	105	105
query13	1653	500	399	399
query14	9664	6437	6166	6166
query15	217	189	176	176
query16	7286	593	477	477
query17	1169	717	550	550
query18	1955	412	302	302
query19	206	190	158	158
query20	121	117	116	116
query21	207	122	104	104
query22	4260	4183	4077	4077
query23	34066	32976	33195	32976
query24	7711	2365	2435	2365
query25	558	498	412	412
query26	1243	271	158	158
query27	2428	477	327	327
query28	4210	2398	2405	2398
query29	755	602	407	407
query30	286	214	190	190
query31	952	853	758	758
query32	70	63	68	63
query33	585	362	301	301
query34	787	847	493	493
query35	806	796	745	745
query36	956	955	897	897
query37	118	102	73	73
query38	4318	4162	4072	4072
query39	1491	1419	1424	1419
query40	204	115	100	100
query41	54	51	54	51
query42	117	103	105	103
query43	491	508	481	481
query44	1273	801	772	772
query45	173	168	164	164
query46	831	1019	626	626
query47	1812	1781	1706	1706
query48	366	398	292	292
query49	802	507	413	413
query50	692	747	397	397
query51	4216	4201	4134	4134
query52	116	109	100	100
query53	226	249	188	188
query54	485	481	421	421
query55	83	83	75	75
query56	281	260	246	246
query57	1114	1150	1074	1074
query58	246	233	270	233
query59	2663	2836	2729	2729
query60	298	267	287	267
query61	119	119	117	117
query62	806	724	691	691
query63	242	192	188	188
query64	4294	996	649	649
query65	4418	4302	4310	4302
query66	1113	401	301	301
query67	15854	15533	15605	15533
query68	8018	878	503	503
query69	458	298	266	266
query70	1230	1165	1081	1081
query71	468	294	260	260
query72	5304	3480	3680	3480
query73	736	682	347	347
query74	8966	9185	8975	8975
query75	3810	3134	2712	2712
query76	3541	1172	728	728
query77	787	363	284	284
query78	10023	10279	9342	9342
query79	1528	882	587	587
query80	661	581	450	450
query81	488	259	219	219
query82	208	129	93	93
query83	175	172	157	157
query84	240	92	73	73
query85	749	345	306	306
query86	384	323	302	302
query87	4572	4566	4294	4294
query88	2872	2255	2238	2238
query89	386	317	290	290
query90	1842	214	221	214
query91	140	139	107	107
query92	66	61	60	60
query93	1184	1056	592	592
query94	589	416	306	306
query95	362	274	264	264
query96	481	560	274	274
query97	3280	3444	3244	3244
query98	222	208	227	208
query99	1305	1393	1285	1285
Total cold run time: 271930 ms
Total hot run time: 184942 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.6 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6a516d1ed9fe329c934eeeeb485136333c5935dc, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.56	0.54	0.55
query6	1.18	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.52	0.51
query10	0.57	0.62	0.57
query11	0.15	0.11	0.10
query12	0.15	0.11	0.12
query13	0.61	0.60	0.60
query14	2.77	2.83	2.70
query15	0.92	0.85	0.86
query16	0.37	0.38	0.39
query17	1.02	1.02	1.01
query18	0.20	0.20	0.20
query19	1.91	1.77	2.02
query20	0.02	0.01	0.02
query21	15.36	0.88	0.54
query22	0.76	1.28	0.71
query23	14.80	1.31	0.67
query24	6.62	1.84	0.55
query25	0.53	0.24	0.08
query26	0.64	0.16	0.13
query27	0.05	0.05	0.04
query28	9.00	0.88	0.44
query29	12.57	3.97	3.31
query30	0.24	0.09	0.06
query31	2.83	0.58	0.39
query32	3.22	0.54	0.46
query33	2.98	3.03	3.00
query34	15.82	5.14	4.51
query35	4.60	4.55	4.60
query36	0.65	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 104.21 s
Total hot run time: 30.6 s

@hubgeter
Copy link
Contributor Author

hubgeter commented Mar 9, 2025

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.70% (17633/26840)
Region Coverage: 65.14% (8690/13340)
Branch Coverage: 55.09% (4688/8510)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a7747ad1ad6a2a47d778e98bd9a92cbe3dfc950c_a7747ad1ad6a2a47d778e98bd9a92cbe3dfc950c_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32648 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a7747ad1ad6a2a47d778e98bd9a92cbe3dfc950c, data reload: false

------ Round 1 ----------------------------------
q1	17668	5349	5175	5175
q2	2060	293	167	167
q3	10799	1262	801	801
q4	10321	1017	546	546
q5	8512	2426	2346	2346
q6	192	168	138	138
q7	909	768	617	617
q8	9327	1285	1095	1095
q9	4906	4666	4861	4666
q10	6817	2319	1885	1885
q11	480	287	264	264
q12	354	356	227	227
q13	17783	3685	3049	3049
q14	239	229	222	222
q15	534	487	489	487
q16	638	622	579	579
q17	585	844	356	356
q18	6733	6562	6391	6391
q19	1680	971	540	540
q20	324	320	198	198
q21	2811	2143	1936	1936
q22	1034	986	963	963
Total cold run time: 104706 ms
Total hot run time: 32648 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5308	5192	5123	5123
q2	249	329	237	237
q3	2194	2655	2341	2341
q4	1417	1833	1427	1427
q5	4306	4148	4168	4148
q6	211	171	126	126
q7	1970	1938	1751	1751
q8	2665	2640	2552	2552
q9	7219	7217	7123	7123
q10	3058	3211	2804	2804
q11	567	523	483	483
q12	673	720	574	574
q13	3430	3958	3284	3284
q14	280	314	279	279
q15	522	486	480	480
q16	686	674	643	643
q17	1155	1580	1372	1372
q18	7754	7668	7479	7479
q19	844	860	932	860
q20	2025	2062	1882	1882
q21	5495	5050	4727	4727
q22	1089	1086	1024	1024
Total cold run time: 53117 ms
Total hot run time: 50719 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191684 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a7747ad1ad6a2a47d778e98bd9a92cbe3dfc950c, data reload: false

query1	1383	1013	1030	1013
query2	6356	1927	1871	1871
query3	11015	4509	4312	4312
query4	54031	24828	23272	23272
query5	5073	581	497	497
query6	346	207	204	204
query7	4921	506	296	296
query8	317	251	242	242
query9	5620	2513	2532	2513
query10	449	310	260	260
query11	15263	15107	14906	14906
query12	164	123	111	111
query13	1061	516	396	396
query14	10649	6733	6593	6593
query15	228	216	173	173
query16	7096	634	478	478
query17	1099	754	582	582
query18	1578	425	310	310
query19	207	194	169	169
query20	128	127	123	123
query21	213	126	112	112
query22	4303	4712	4438	4438
query23	34059	33453	33265	33265
query24	5986	2493	2446	2446
query25	460	471	404	404
query26	709	277	153	153
query27	1927	500	340	340
query28	2771	2442	2431	2431
query29	609	562	430	430
query30	271	226	188	188
query31	895	859	817	817
query32	76	64	65	64
query33	493	352	297	297
query34	946	867	497	497
query35	801	841	743	743
query36	976	988	883	883
query37	119	95	77	77
query38	4250	4277	4299	4277
query39	1518	1417	1458	1417
query40	205	113	105	105
query41	52	50	49	49
query42	122	108	110	108
query43	500	507	486	486
query44	1307	817	806	806
query45	183	179	167	167
query46	846	1048	667	667
query47	1866	1864	1778	1778
query48	387	422	306	306
query49	667	531	482	482
query50	704	744	413	413
query51	4307	4307	4281	4281
query52	130	110	97	97
query53	228	267	201	201
query54	508	495	415	415
query55	85	84	81	81
query56	276	283	246	246
query57	1162	1200	1120	1120
query58	246	302	234	234
query59	2639	2926	2769	2769
query60	288	258	266	258
query61	117	114	118	114
query62	731	745	679	679
query63	237	196	186	186
query64	1961	1021	683	683
query65	4490	4510	4410	4410
query66	704	390	312	312
query67	15699	15504	15226	15226
query68	7247	866	503	503
query69	531	292	278	278
query70	1188	1111	1018	1018
query71	527	338	263	263
query72	5757	3595	3706	3595
query73	1465	757	347	347
query74	9287	8985	8677	8677
query75	3842	3174	2678	2678
query76	4220	1177	739	739
query77	650	350	280	280
query78	10075	10031	9253	9253
query79	2650	795	587	587
query80	702	522	443	443
query81	478	260	219	219
query82	647	126	95	95
query83	268	169	152	152
query84	289	88	74	74
query85	781	340	324	324
query86	360	305	279	279
query87	4490	4556	4532	4532
query88	3550	2221	2221	2221
query89	408	313	275	275
query90	1914	209	205	205
query91	156	138	107	107
query92	76	62	57	57
query93	1639	1053	583	583
query94	660	420	294	294
query95	348	268	255	255
query96	492	555	269	269
query97	3344	3409	3309	3309
query98	223	207	210	207
query99	1412	1424	1287	1287
Total cold run time: 298689 ms
Total hot run time: 191684 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.35 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a7747ad1ad6a2a47d778e98bd9a92cbe3dfc950c, data reload: false

query1	0.03	0.02	0.03
query2	0.11	0.05	0.05
query3	0.28	0.05	0.06
query4	1.60	0.08	0.07
query5	0.56	0.56	0.53
query6	1.20	0.73	0.73
query7	0.02	0.01	0.01
query8	0.07	0.05	0.05
query9	0.63	0.52	0.53
query10	0.59	0.59	0.58
query11	0.26	0.14	0.13
query12	0.26	0.13	0.13
query13	0.64	0.62	0.61
query14	2.71	2.68	2.70
query15	1.00	0.87	0.86
query16	0.37	0.37	0.38
query17	1.04	1.04	1.06
query18	0.18	0.19	0.18
query19	2.05	1.93	1.87
query20	0.02	0.01	0.01
query21	15.36	0.96	0.66
query22	0.93	1.04	0.82
query23	14.70	1.53	0.75
query24	5.31	0.62	0.29
query25	0.17	0.09	0.09
query26	0.55	0.22	0.18
query27	0.08	0.08	0.08
query28	11.06	1.16	0.56
query29	12.56	4.12	3.37
query30	0.27	0.08	0.06
query31	2.85	0.65	0.42
query32	3.23	0.59	0.50
query33	3.04	3.01	3.06
query34	16.65	5.14	4.43
query35	4.53	4.49	4.46
query36	0.64	0.51	0.51
query37	0.21	0.18	0.17
query38	0.16	0.16	0.15
query39	0.05	0.05	0.04
query40	0.20	0.17	0.15
query41	0.11	0.06	0.06
query42	0.06	0.05	0.05
query43	0.05	0.05	0.06
Total cold run time: 106.39 s
Total hot run time: 31.35 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 26.73% (81/303) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 46.89% (12544/26750)
Line Coverage 36.48% (106991/293290)
Region Coverage 35.52% (54635/153834)
Branch Coverage 30.86% (27472/89026)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants