Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](group commit)Fix wal manager ut #48789

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hust-hhb
Copy link
Contributor

@hust-hhb hust-hhb commented Mar 6, 2025

What problem does this PR solve?

  1. remove invalid ut WalManagerTest.recovery_normal, this is a fake ut because it doesn't recover wal really, and now we have regression test to do real recovery
  2. merge two ut file into one
  3. fix heap use after free problem

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Mar 6, 2025

run buildall

1 similar comment
@hust-hhb
Copy link
Contributor Author

hust-hhb commented Mar 6, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32552 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9505884568ad6dd8075f716c4a605fe8465e883, data reload: false

------ Round 1 ----------------------------------
q1	17578	5228	5094	5094
q2	2054	311	170	170
q3	10399	1326	701	701
q4	10208	1027	517	517
q5	7545	2442	2381	2381
q6	188	165	131	131
q7	895	742	598	598
q8	9301	1253	1105	1105
q9	4878	4797	4728	4728
q10	6827	2313	1931	1931
q11	470	276	265	265
q12	343	355	213	213
q13	17750	3638	3067	3067
q14	232	230	210	210
q15	552	507	496	496
q16	613	605	594	594
q17	562	919	351	351
q18	6820	6369	6362	6362
q19	1221	937	526	526
q20	308	339	194	194
q21	2781	2123	1939	1939
q22	1082	1024	979	979
Total cold run time: 102607 ms
Total hot run time: 32552 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5202	5095	5037	5037
q2	236	327	238	238
q3	2173	2682	2302	2302
q4	1490	1845	1379	1379
q5	4256	4067	4168	4067
q6	205	161	126	126
q7	1866	1882	1798	1798
q8	2561	2617	2550	2550
q9	7163	7105	7223	7105
q10	3007	3178	2732	2732
q11	586	518	498	498
q12	692	764	592	592
q13	3543	3877	3301	3301
q14	291	302	259	259
q15	539	505	485	485
q16	638	706	658	658
q17	1130	1604	1320	1320
q18	7765	7596	7406	7406
q19	859	900	1073	900
q20	1971	2004	1864	1864
q21	5499	5073	4856	4856
q22	1107	1076	1010	1010
Total cold run time: 52779 ms
Total hot run time: 50483 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184431 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9505884568ad6dd8075f716c4a605fe8465e883, data reload: false

query1	986	391	393	391
query2	6532	1922	1979	1922
query3	6794	217	217	217
query4	26530	23671	22930	22930
query5	4324	658	472	472
query6	311	214	195	195
query7	4595	496	296	296
query8	294	238	230	230
query9	8599	2555	2561	2555
query10	481	315	262	262
query11	15910	14989	14842	14842
query12	166	105	101	101
query13	1649	501	377	377
query14	9759	6071	6384	6071
query15	212	192	167	167
query16	7648	626	464	464
query17	1161	671	574	574
query18	1976	382	292	292
query19	185	178	146	146
query20	119	112	110	110
query21	208	119	101	101
query22	4298	4294	4219	4219
query23	33888	33118	33003	33003
query24	8103	2398	2417	2398
query25	530	473	422	422
query26	1237	271	157	157
query27	2674	487	327	327
query28	4344	2392	2388	2388
query29	710	584	424	424
query30	287	219	190	190
query31	950	846	758	758
query32	70	66	63	63
query33	555	358	294	294
query34	769	841	499	499
query35	782	822	720	720
query36	950	979	909	909
query37	120	95	76	76
query38	4101	4270	4019	4019
query39	1438	1388	1467	1388
query40	206	114	104	104
query41	56	54	52	52
query42	118	105	106	105
query43	523	530	488	488
query44	1278	796	797	796
query45	175	170	171	170
query46	820	1015	621	621
query47	1765	1837	1673	1673
query48	406	408	293	293
query49	788	531	432	432
query50	666	716	422	422
query51	4211	4217	4095	4095
query52	107	101	93	93
query53	245	254	188	188
query54	480	489	424	424
query55	84	79	82	79
query56	274	270	246	246
query57	1144	1146	1072	1072
query58	239	244	238	238
query59	2714	2650	2688	2650
query60	281	279	268	268
query61	123	126	142	126
query62	810	744	707	707
query63	238	197	198	197
query64	4354	1091	744	744
query65	4501	4324	4365	4324
query66	1040	408	294	294
query67	15663	15463	15120	15120
query68	7875	880	513	513
query69	455	296	262	262
query70	1175	1163	1112	1112
query71	455	285	273	273
query72	5544	3499	3679	3499
query73	769	703	347	347
query74	9213	9105	8757	8757
query75	3805	3157	2655	2655
query76	3677	1169	742	742
query77	803	365	289	289
query78	9950	10125	9266	9266
query79	2221	818	602	602
query80	609	519	450	450
query81	463	260	218	218
query82	543	125	97	97
query83	183	177	161	161
query84	232	94	74	74
query85	795	354	381	354
query86	336	304	279	279
query87	4515	4600	4379	4379
query88	3663	2229	2244	2229
query89	390	314	282	282
query90	1966	210	212	210
query91	145	140	112	112
query92	74	60	58	58
query93	1577	1056	578	578
query94	674	406	313	313
query95	352	279	264	264
query96	480	566	273	273
query97	3350	3428	3329	3329
query98	245	210	200	200
query99	1635	1371	1264	1264
Total cold run time: 275723 ms
Total hot run time: 184431 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9505884568ad6dd8075f716c4a605fe8465e883, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.60	0.11	0.11
query5	0.56	0.55	0.54
query6	1.20	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.51
query10	0.57	0.63	0.58
query11	0.16	0.11	0.11
query12	0.14	0.12	0.11
query13	0.61	0.62	0.59
query14	2.72	2.68	2.68
query15	0.92	0.86	0.85
query16	0.37	0.37	0.38
query17	1.05	1.07	1.04
query18	0.21	0.19	0.19
query19	1.92	1.80	1.99
query20	0.02	0.02	0.01
query21	15.36	0.88	0.54
query22	0.77	1.18	0.63
query23	14.96	1.37	0.58
query24	6.88	1.12	1.49
query25	0.54	0.21	0.20
query26	0.56	0.16	0.14
query27	0.06	0.05	0.05
query28	9.92	0.85	0.43
query29	12.53	4.13	3.43
query30	0.25	0.09	0.07
query31	2.83	0.60	0.40
query32	3.23	0.55	0.48
query33	2.97	3.02	2.98
query34	15.85	5.13	4.57
query35	4.54	4.54	4.51
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 105.45 s
Total hot run time: 31.31 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 46.71% (12492/26745)
Line Coverage 36.29% (106349/293048)
Region Coverage 35.36% (54356/153724)
Branch Coverage 30.73% (27333/88936)

hello-stephen
hello-stephen previously approved these changes Mar 7, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 7, 2025
Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by anyone and no changes requested.

zhangstar333
zhangstar333 previously approved these changes Mar 7, 2025
zhannngchen
zhannngchen previously approved these changes Mar 7, 2025
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to resolve conflict

@hust-hhb
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Mar 10, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 32566 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b9bb046418323e8438a6bf6f3d731f65887bc51, data reload: false

------ Round 1 ----------------------------------
q1	17604	5474	5165	5165
q2	2049	293	171	171
q3	10400	1322	718	718
q4	10228	1039	530	530
q5	7554	2371	2357	2357
q6	187	169	138	138
q7	897	732	601	601
q8	9280	1278	1176	1176
q9	5126	4828	4615	4615
q10	6828	2321	1892	1892
q11	461	279	266	266
q12	345	348	209	209
q13	17778	3695	3070	3070
q14	234	224	208	208
q15	532	491	475	475
q16	621	630	584	584
q17	569	883	343	343
q18	7091	6495	6452	6452
q19	1207	945	542	542
q20	316	324	190	190
q21	2815	2138	1887	1887
q22	1075	1036	977	977
Total cold run time: 103197 ms
Total hot run time: 32566 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5246	5179	5177	5177
q2	244	331	228	228
q3	2167	2660	2318	2318
q4	1466	1835	1392	1392
q5	4254	4135	4189	4135
q6	208	162	123	123
q7	1900	1952	1809	1809
q8	2591	2753	2642	2642
q9	7186	7272	7170	7170
q10	3002	3199	2754	2754
q11	590	525	509	509
q12	685	762	594	594
q13	3394	3863	3198	3198
q14	277	305	264	264
q15	527	466	476	466
q16	665	691	668	668
q17	1162	1554	1344	1344
q18	7853	7681	7543	7543
q19	795	845	925	845
q20	2010	1986	1850	1850
q21	5406	4839	4929	4839
q22	1054	1037	999	999
Total cold run time: 52682 ms
Total hot run time: 50867 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185372 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7b9bb046418323e8438a6bf6f3d731f65887bc51, data reload: false

query1	994	383	375	375
query2	6518	1881	1895	1881
query3	6797	231	221	221
query4	26094	23816	23492	23492
query5	4279	626	475	475
query6	300	193	209	193
query7	4614	486	296	296
query8	299	254	239	239
query9	8607	2533	2518	2518
query10	465	312	251	251
query11	15640	15253	15022	15022
query12	161	107	107	107
query13	1661	526	412	412
query14	9499	6073	6094	6073
query15	204	190	175	175
query16	7295	592	474	474
query17	1239	728	565	565
query18	1960	420	308	308
query19	200	191	162	162
query20	119	115	115	115
query21	207	122	103	103
query22	4343	4521	4367	4367
query23	33870	33076	32832	32832
query24	7657	2349	2388	2349
query25	526	449	373	373
query26	1230	275	158	158
query27	2230	479	329	329
query28	4090	2442	2374	2374
query29	754	562	419	419
query30	283	211	193	193
query31	926	894	757	757
query32	76	67	61	61
query33	552	350	303	303
query34	790	864	501	501
query35	786	811	740	740
query36	953	987	895	895
query37	125	95	79	79
query38	4186	4098	4265	4098
query39	1451	1414	1398	1398
query40	202	114	99	99
query41	55	53	52	52
query42	121	105	101	101
query43	508	503	487	487
query44	1300	794	793	793
query45	172	167	167	167
query46	821	1016	630	630
query47	1737	1813	1721	1721
query48	369	406	295	295
query49	792	527	410	410
query50	679	746	402	402
query51	4188	4168	4125	4125
query52	102	102	100	100
query53	225	247	181	181
query54	483	476	427	427
query55	83	84	80	80
query56	267	267	254	254
query57	1125	1148	1089	1089
query58	249	236	240	236
query59	2606	2681	2484	2484
query60	277	262	258	258
query61	127	117	121	117
query62	811	740	667	667
query63	234	186	187	186
query64	4383	1050	678	678
query65	4409	4334	4370	4334
query66	1149	415	311	311
query67	15739	15473	15210	15210
query68	8176	875	513	513
query69	474	303	258	258
query70	1141	1071	1116	1071
query71	445	287	268	268
query72	5597	3518	3660	3518
query73	780	723	345	345
query74	9285	8919	8915	8915
query75	3697	3134	2732	2732
query76	3611	1180	742	742
query77	776	388	293	293
query78	9974	10202	9425	9425
query79	2597	835	591	591
query80	620	516	445	445
query81	466	254	230	230
query82	663	128	96	96
query83	204	179	227	179
query84	288	90	75	75
query85	779	347	302	302
query86	331	312	293	293
query87	4535	4491	4377	4377
query88	3442	2185	2216	2185
query89	386	320	286	286
query90	1919	211	210	210
query91	133	146	115	115
query92	82	62	60	60
query93	1466	1042	583	583
query94	657	400	286	286
query95	361	265	260	260
query96	476	562	273	273
query97	3339	3432	3331	3331
query98	235	213	217	213
query99	1426	1407	1325	1325
Total cold run time: 273554 ms
Total hot run time: 185372 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7b9bb046418323e8438a6bf6f3d731f65887bc51, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.64	0.10	0.10
query5	0.57	0.55	0.54
query6	1.22	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.53	0.52
query10	0.57	0.60	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.61	0.60
query14	2.79	2.80	2.72
query15	0.92	0.86	0.85
query16	0.39	0.37	0.37
query17	1.01	1.04	1.05
query18	0.21	0.19	0.19
query19	1.97	1.76	2.02
query20	0.01	0.02	0.01
query21	15.36	0.93	0.55
query22	0.76	1.25	0.66
query23	14.89	1.41	0.61
query24	7.64	1.30	0.83
query25	0.52	0.29	0.12
query26	0.58	0.16	0.14
query27	0.05	0.04	0.05
query28	8.98	0.85	0.44
query29	12.55	4.02	3.28
query30	0.25	0.09	0.07
query31	2.83	0.61	0.39
query32	3.22	0.55	0.46
query33	3.01	2.99	2.98
query34	15.77	5.02	4.46
query35	4.56	4.51	4.53
query36	0.67	0.50	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.31 s
Total hot run time: 30.73 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants