Skip to content

branch-3.1: [fix](hdfs) remove cached file handle when read fails #54926 #54992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Aug 20, 2025
Merged

branch-3.1: [fix](hdfs) remove cached file handle when read fails #54926 #54992

morrySnow merged 1 commit into from
Aug 20, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 19, 2025

Cherry-picked from #54926

@github-actions github-actions bot requested a review from morrySnow as a code owner August 19, 2025 07:07
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 19, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Aug 19, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 19, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 19, 2025

TPC-H: Total hot run time: 32510 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 226b2f965f43a4c57d8fa774469dc6ca91662fa8, data reload: false

------ Round 1 ----------------------------------
q1	17581	5441	5425	5425
q2	2036	381	287	287
q3	12144	1232	730	730
q4	10239	864	446	446
q5	8430	2381	2100	2100
q6	183	163	135	135
q7	894	746	612	612
q8	9328	1422	1150	1150
q9	5218	4972	4901	4901
q10	6794	2279	1802	1802
q11	483	282	257	257
q12	334	355	216	216
q13	17764	3596	2973	2973
q14	237	224	215	215
q15	521	478	458	458
q16	416	423	373	373
q17	593	855	354	354
q18	6832	6441	6352	6352
q19	1237	959	553	553
q20	326	337	206	206
q21	2868	2184	1961	1961
q22	1035	1023	1004	1004
Total cold run time: 105493 ms
Total hot run time: 32510 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5500	5470	5483	5470
q2	228	325	237	237
q3	2254	2645	2322	2322
q4	1338	1753	1385	1385
q5	4360	4884	4919	4884
q6	176	167	127	127
q7	2025	1972	1865	1865
q8	2610	2823	2667	2667
q9	7289	7226	7216	7216
q10	3016	3309	2716	2716
q11	562	505	485	485
q12	639	756	620	620
q13	3414	3742	3146	3146
q14	278	295	274	274
q15	522	468	473	468
q16	434	496	431	431
q17	1226	1749	1280	1280
q18	7539	7558	7287	7287
q19	808	932	1182	932
q20	2056	2052	1880	1880
q21	5270	4782	4551	4551
q22	1059	1083	997	997
Total cold run time: 52603 ms
Total hot run time: 51240 ms

@doris-robot
Copy link

doris-robot commented Aug 19, 2025

TPC-DS: Total hot run time: 191766 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 226b2f965f43a4c57d8fa774469dc6ca91662fa8, data reload: false

query1	966	408	409	408
query2	6249	1961	1903	1903
query3	8679	195	195	195
query4	33615	23798	23304	23304
query5	3654	582	448	448
query6	281	202	173	173
query7	4196	480	310	310
query8	302	240	243	240
query9	9475	2605	2573	2573
query10	495	349	254	254
query11	18220	15457	15151	15151
query12	161	109	102	102
query13	1565	562	419	419
query14	9985	7228	6693	6693
query15	266	202	182	182
query16	8050	665	513	513
query17	1612	760	601	601
query18	2145	424	335	335
query19	234	200	200	200
query20	133	135	132	132
query21	212	132	107	107
query22	4555	4522	4420	4420
query23	35168	34029	33729	33729
query24	7238	2719	2730	2719
query25	559	480	428	428
query26	785	295	175	175
query27	2180	488	375	375
query28	5174	2192	2163	2163
query29	690	597	460	460
query30	236	191	157	157
query31	1014	927	845	845
query32	94	68	64	64
query33	527	376	340	340
query34	746	848	525	525
query35	767	816	738	738
query36	1017	1077	972	972
query37	111	95	68	68
query38	4080	4029	4014	4014
query39	1544	1456	1495	1456
query40	209	115	108	108
query41	139	49	49	49
query42	121	108	111	108
query43	521	523	479	479
query44	1369	836	828	828
query45	194	176	176	176
query46	864	1060	688	688
query47	1996	1984	1953	1953
query48	405	421	348	348
query49	739	486	443	443
query50	673	695	442	442
query51	7378	7429	7321	7321
query52	106	106	94	94
query53	232	255	197	197
query54	557	576	459	459
query55	81	78	83	78
query56	269	268	275	268
query57	1320	1262	1223	1223
query58	237	212	218	212
query59	3146	3324	3019	3019
query60	301	320	268	268
query61	114	107	111	107
query62	828	754	685	685
query63	236	195	200	195
query64	3672	976	647	647
query65	3367	3279	3192	3192
query66	773	408	300	300
query67	16112	15750	15586	15586
query68	7575	828	541	541
query69	495	300	262	262
query70	1177	1142	1165	1142
query71	370	281	257	257
query72	5830	3758	3881	3758
query73	645	765	361	361
query74	10407	9289	9054	9054
query75	3184	3145	2655	2655
query76	2954	1168	759	759
query77	460	354	278	278
query78	10515	10731	9794	9794
query79	3337	923	580	580
query80	625	547	438	438
query81	506	256	221	221
query82	610	119	90	90
query83	168	160	143	143
query84	243	107	77	77
query85	802	349	307	307
query86	405	309	304	304
query87	4307	4356	4288	4288
query88	5150	2407	2387	2387
query89	421	347	301	301
query90	1790	196	194	194
query91	143	139	112	112
query92	65	57	52	52
query93	2089	919	542	542
query94	701	428	295	295
query95	339	284	272	272
query96	498	625	286	286
query97	3210	3253	3162	3162
query98	228	211	196	196
query99	1368	1407	1294	1294
Total cold run time: 293556 ms
Total hot run time: 191766 ms

@doris-robot
Copy link

doris-robot commented Aug 19, 2025

ClickBench: Total hot run time: 28.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 226b2f965f43a4c57d8fa774469dc6ca91662fa8, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.11	0.11
query5	0.52	0.52	0.50
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.59	0.50	0.49
query10	0.57	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.60	0.58
query14	0.76	0.80	0.81
query15	0.85	0.83	0.83
query16	0.40	0.39	0.39
query17	1.07	1.02	1.03
query18	0.25	0.22	0.23
query19	1.96	1.85	1.79
query20	0.01	0.02	0.01
query21	15.37	0.93	0.59
query22	0.75	0.88	0.53
query23	15.16	1.43	0.52
query24	3.44	1.13	0.72
query25	0.18	0.16	0.16
query26	0.31	0.15	0.14
query27	0.06	0.04	0.05
query28	12.85	1.00	0.43
query29	12.54	3.99	3.26
query30	0.25	0.09	0.06
query31	2.83	0.62	0.38
query32	3.22	0.54	0.45
query33	3.03	3.02	3.02
query34	16.58	5.20	4.50
query35	4.52	4.52	4.57
query36	0.62	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.23 s
Total hot run time: 28.5 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 19, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.49% (12723/27970)
Line Coverage 36.37% (113414/311807)
Region Coverage 34.01% (64915/190882)
Branch Coverage 31.04% (34072/109758)

@morrySnow morrySnow merged commit 56e2269 into branch-3.1 Aug 20, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-54926-branch-3.1 branch August 20, 2025 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @morrySnow @dataroaring