Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](inverted index) Added Error Check for Zero File Read Result #48666

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zzzxl1993
Copy link
Contributor

@zzzxl1993 zzzxl1993 commented Mar 5, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

  1. "When running the test_index_compound_directory_fault_injection case with HDFS for remote storage, issues may occur. Added error handling for cases where reading a file returns a value of 0."

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32613 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 71c88f94acd6db6be9dc49bf0fd2b85f07dc7f05, data reload: false

------ Round 1 ----------------------------------
q1	17616	5312	5106	5106
q2	2062	321	172	172
q3	10422	1273	759	759
q4	10214	1042	535	535
q5	7501	2384	2396	2384
q6	188	174	143	143
q7	927	781	601	601
q8	9310	1381	1100	1100
q9	4923	4706	4646	4646
q10	6842	2339	1891	1891
q11	476	271	261	261
q12	352	359	212	212
q13	17761	3746	3120	3120
q14	224	226	211	211
q15	543	504	492	492
q16	640	620	595	595
q17	592	863	348	348
q18	6944	6417	6365	6365
q19	1925	954	544	544
q20	321	320	192	192
q21	2915	2114	1956	1956
q22	1084	1057	980	980
Total cold run time: 103782 ms
Total hot run time: 32613 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5242	5132	5163	5132
q2	239	337	238	238
q3	2162	2656	2286	2286
q4	1436	1900	1451	1451
q5	4295	4177	4183	4177
q6	211	168	132	132
q7	1941	1929	1784	1784
q8	2655	2661	2591	2591
q9	7376	7204	7223	7204
q10	3036	3236	2802	2802
q11	572	520	493	493
q12	668	727	569	569
q13	3455	3909	3319	3319
q14	281	300	269	269
q15	555	496	514	496
q16	679	705	652	652
q17	1140	1596	1345	1345
q18	7919	7711	7453	7453
q19	811	827	996	827
q20	1973	2028	1876	1876
q21	5474	4987	4817	4817
q22	1146	1107	1065	1065
Total cold run time: 53266 ms
Total hot run time: 50978 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191440 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 71c88f94acd6db6be9dc49bf0fd2b85f07dc7f05, data reload: false

query1	1436	1041	1029	1029
query2	6306	1849	1857	1849
query3	11139	4473	4597	4473
query4	25602	23744	23166	23166
query5	3830	648	475	475
query6	279	200	187	187
query7	3987	499	286	286
query8	301	248	248	248
query9	8486	2508	2516	2508
query10	453	315	250	250
query11	16137	15030	14904	14904
query12	184	116	108	108
query13	1579	527	394	394
query14	9218	6271	6448	6271
query15	216	189	178	178
query16	7453	661	445	445
query17	1205	795	597	597
query18	2009	428	334	334
query19	204	193	173	173
query20	128	123	131	123
query21	208	125	111	111
query22	4485	4539	4514	4514
query23	34438	33701	33626	33626
query24	7783	2436	2453	2436
query25	519	489	418	418
query26	1286	271	150	150
query27	2449	478	329	329
query28	4696	2455	2421	2421
query29	715	596	438	438
query30	282	232	193	193
query31	918	869	782	782
query32	72	66	61	61
query33	527	360	306	306
query34	791	848	495	495
query35	831	847	759	759
query36	971	975	893	893
query37	113	96	79	79
query38	4110	4137	4141	4137
query39	1513	1424	1441	1424
query40	212	122	103	103
query41	76	51	51	51
query42	126	108	103	103
query43	491	505	469	469
query44	1305	788	790	788
query45	183	172	169	169
query46	870	1030	632	632
query47	1843	1906	1813	1813
query48	383	425	309	309
query49	770	517	433	433
query50	707	755	417	417
query51	4312	4291	4267	4267
query52	107	102	98	98
query53	236	266	189	189
query54	517	488	428	428
query55	81	87	88	87
query56	273	265	253	253
query57	1165	1206	1093	1093
query58	285	244	237	237
query59	2720	2796	2570	2570
query60	301	275	252	252
query61	122	121	119	119
query62	791	745	672	672
query63	225	196	192	192
query64	4030	1055	665	665
query65	4499	4408	4458	4408
query66	999	405	301	301
query67	16633	15776	15468	15468
query68	9711	879	554	554
query69	482	300	271	271
query70	1195	1129	1097	1097
query71	463	283	278	278
query72	5249	3537	3834	3537
query73	787	737	342	342
query74	8983	8884	8699	8699
query75	4187	3210	2692	2692
query76	3693	1168	745	745
query77	982	360	287	287
query78	9945	10022	9324	9324
query79	4299	815	564	564
query80	625	519	449	449
query81	464	255	221	221
query82	234	126	90	90
query83	162	167	162	162
query84	299	98	76	76
query85	751	355	301	301
query86	335	312	300	300
query87	4548	4535	4372	4372
query88	2783	2197	2151	2151
query89	433	315	271	271
query90	2172	191	189	189
query91	134	146	105	105
query92	72	57	56	56
query93	2685	1041	563	563
query94	673	414	287	287
query95	347	268	251	251
query96	490	541	264	264
query97	3340	3401	3294	3294
query98	222	211	205	205
query99	1390	1439	1287	1287
Total cold run time: 283256 ms
Total hot run time: 191440 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 71c88f94acd6db6be9dc49bf0fd2b85f07dc7f05, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.55	0.56	0.57
query6	1.17	0.72	0.72
query7	0.03	0.01	0.02
query8	0.04	0.04	0.03
query9	0.58	0.55	0.52
query10	0.56	0.57	0.57
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.66	2.83	2.80
query15	0.93	0.85	0.85
query16	0.37	0.39	0.38
query17	1.00	1.01	1.04
query18	0.20	0.20	0.19
query19	1.92	1.97	1.85
query20	0.02	0.02	0.01
query21	15.35	0.87	0.55
query22	0.77	1.20	0.72
query23	14.89	1.38	0.64
query24	6.55	1.73	1.32
query25	0.51	0.32	0.05
query26	0.53	0.16	0.14
query27	0.05	0.05	0.05
query28	9.85	0.85	0.44
query29	12.66	3.96	3.23
query30	0.25	0.09	0.06
query31	2.83	0.57	0.38
query32	3.25	0.55	0.46
query33	3.12	3.00	2.99
query34	15.77	5.21	4.52
query35	4.54	4.53	4.55
query36	0.65	0.49	0.50
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.97 s
Total hot run time: 31.44 s

Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 5, 2025
Copy link
Contributor

github-actions bot commented Mar 5, 2025

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Mar 5, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 14.29% (1/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.82% (12245/26722)
Line Coverage 35.36% (103565/292848)
Region Coverage 34.53% (53067/153668)
Branch Coverage 30.24% (26888/88908)

@zzzxl1993 zzzxl1993 marked this pull request as draft March 5, 2025 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants