Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](excutor) Fix high priority push task thread hold lock long time #48259

Open
wants to merge 1 commit into
base: branch-2.0
Choose a base branch
from

Conversation

liutang123
Copy link
Contributor

What problem does this PR solve?

Same as #48177

Intruduced by #23538
Sleep after release the lock.

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@liutang123
Copy link
Contributor Author

run buildall

@Thearas
Copy link
Contributor

Thearas commented Feb 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 49095 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 398fc86852fd82ab1e80884ef3acba26290314e6, data reload: false

------ Round 1 ----------------------------------
q1	17777	4398	4417	4398
q2	2033	154	142	142
q3	10324	1918	1939	1918
q4	10112	1215	1360	1215
q5	8456	3926	3920	3920
q6	229	125	146	125
q7	2030	1622	1625	1622
q8	9312	2732	2736	2732
q9	10373	10029	9719	9719
q10	8648	3610	3567	3567
q11	425	238	245	238
q12	469	301	297	297
q13	18383	3991	4064	3991
q14	355	329	337	329
q15	495	463	457	457
q16	539	459	459	459
q17	1127	947	946	946
q18	7261	6740	6800	6740
q19	1703	1544	1489	1489
q20	547	297	297	297
q21	4414	4116	4098	4098
q22	480	404	396	396
Total cold run time: 115492 ms
Total hot run time: 49095 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4344	4371	4296	4296
q2	329	226	234	226
q3	4207	4162	4160	4160
q4	2747	2753	2761	2753
q5	7162	7120	7111	7111
q6	240	124	119	119
q7	3214	2847	2822	2822
q8	4398	4478	4528	4478
q9	13613	13597	13674	13597
q10	4258	4293	4280	4280
q11	752	676	671	671
q12	1045	886	871	871
q13	6556	3735	3743	3735
q14	456	431	422	422
q15	511	470	461	461
q16	628	571	576	571
q17	3849	3817	3913	3817
q18	8757	8680	8838	8680
q19	1708	1660	1670	1660
q20	2373	2148	2115	2115
q21	8499	8584	8511	8511
q22	1033	927	925	925
Total cold run time: 80679 ms
Total hot run time: 76281 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.56% (8312/21555)
Line Coverage: 30.26% (68763/227217)
Region Coverage: 29.69% (35409/119276)
Branch Coverage: 25.45% (18203/71530)
Coverage Report: http://coverage.selectdb-in.cc/coverage/398fc86852fd82ab1e80884ef3acba26290314e6_398fc86852fd82ab1e80884ef3acba26290314e6/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 213251 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 398fc86852fd82ab1e80884ef3acba26290314e6, data reload: false

query1	944	422	382	382
query2	6558	2178	2088	2088
query3	6923	200	202	200
query4	23803	21726	21761	21726
query5	19738	6499	6506	6499
query6	284	218	226	218
query7	4348	310	314	310
query8	261	235	239	235
query9	3092	2673	2603	2603
query10	482	315	302	302
query11	15430	15563	14924	14924
query12	133	84	72	72
query13	1057	451	444	444
query14	17293	13590	13623	13590
query15	380	221	233	221
query16	5887	284	263	263
query17	1696	911	924	911
query18	887	340	321	321
query19	221	159	151	151
query20	100	98	109	98
query21	190	98	93	93
query22	5317	5136	5129	5129
query23	34218	33363	33582	33363
query24	7902	6237	6348	6237
query25	522	451	438	438
query26	1333	169	164	164
query27	2438	301	298	298
query28	6191	2281	2281	2281
query29	2866	2724	2763	2724
query30	248	168	167	167
query31	946	751	750	750
query32	70	61	59	59
query33	453	267	248	248
query34	864	470	487	470
query35	1118	923	961	923
query36	1259	1085	1128	1085
query37	171	63	59	59
query38	3087	2949	2947	2947
query39	1370	1340	1323	1323
query40	314	101	95	95
query41	50	46	37	37
query42	83	88	90	88
query43	585	612	580	580
query44	1176	712	725	712
query45	245	231	231	231
query46	1251	976	947	947
query47	1953	1653	1779	1653
query48	508	407	413	407
query49	653	373	386	373
query50	870	578	570	570
query51	4755	4619	4669	4619
query52	87	94	94	94
query53	227	189	198	189
query54	2667	2452	2486	2452
query55	86	78	87	78
query56	227	208	213	208
query57	1317	1135	1230	1135
query58	230	213	209	209
query59	3502	3458	3158	3158
query60	229	216	212	212
query61	95	91	95	91
query62	808	500	489	489
query63	199	178	177	177
query64	3533	1584	1515	1515
query65	3611	3537	3536	3536
query66	782	422	416	416
query67	16076	15669	15457	15457
query68	10482	667	635	635
query69	518	302	270	270
query70	1629	1506	1378	1378
query71	412	327	311	311
query72	6865	4730	4745	4730
query73	760	313	318	313
query74	6440	5841	5934	5841
query75	5367	3799	3759	3759
query76	6118	1182	1216	1182
query77	1019	267	260	260
query78	12920	12032	12015	12015
query79	7664	653	657	653
query80	1471	417	400	400
query81	482	235	238	235
query82	1648	100	96	96
query83	169	131	129	129
query84	261	72	69	69
query85	889	315	321	315
query86	328	301	306	301
query87	3250	3068	2994	2994
query88	4435	2313	2341	2313
query89	490	297	328	297
query90	1944	204	213	204
query91	170	126	133	126
query92	58	51	51	51
query93	6630	584	592	584
query94	705	207	210	207
query95	1999	1992	1738	1738
query96	665	328	328	328
query97	6381	6383	6428	6383
query98	227	218	206	206
query99	2809	909	884	884
Total cold run time: 321404 ms
Total hot run time: 213251 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 398fc86852fd82ab1e80884ef3acba26290314e6, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.75	0.07	0.06
query5	0.53	0.53	0.53
query6	1.24	0.62	0.61
query7	0.02	0.00	0.01
query8	0.04	0.02	0.03
query9	0.51	0.48	0.47
query10	0.58	0.53	0.52
query11	0.12	0.08	0.08
query12	0.12	0.09	0.10
query13	0.62	0.61	0.61
query14	0.79	0.81	0.79
query15	0.78	0.77	0.76
query16	0.38	0.36	0.38
query17	1.01	1.00	1.03
query18	0.19	0.26	0.25
query19	1.93	1.78	1.87
query20	0.01	0.01	0.01
query21	15.46	0.55	0.55
query22	2.11	2.18	1.81
query23	17.23	1.12	0.87
query24	6.39	0.57	1.06
query25	0.37	0.13	0.05
query26	0.60	0.15	0.15
query27	0.05	0.03	0.05
query28	7.09	0.73	0.77
query29	12.76	2.29	2.30
query30	0.59	0.55	0.53
query31	2.81	0.39	0.38
query32	3.37	0.50	0.50
query33	3.06	3.08	3.05
query34	15.26	4.78	4.79
query35	4.82	4.88	4.86
query36	1.06	1.02	1.00
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.39 s
Total hot run time: 30.43 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 398fc86852fd82ab1e80884ef3acba26290314e6 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

Copy link
Member

@xy720 xy720 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@liutang123
Copy link
Contributor Author

run p0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants