Skip to content

branch-3.1: [fix](filecache) unify endian operations and thread renaming method #53140 #53969 #54019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 2 commits into from
Jul 30, 2025
Merged

branch-3.1: [fix](filecache) unify endian operations and thread renaming method #53140 #53969 #54019

morrySnow merged 2 commits into from
Jul 30, 2025

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Jul 29, 2025 •
edited by morrySnow
Loading

picked from #53140 #53969

@freemandealer freemandealer requested a review from morrySnow as a code owner July 29, 2025 07:15
@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor Author

freemandealer commented Jul 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 29, 2025

TPC-H: Total hot run time: 32273 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 808b49e691e65ffb536d2457da9aac5a90dac7c6, data reload: false

------ Round 1 ----------------------------------
q1	17583	5452	5385	5385
q2	2060	281	160	160
q3	10472	1252	742	742
q4	10220	871	446	446
q5	7808	2359	2081	2081
q6	176	161	131	131
q7	882	755	615	615
q8	9315	1407	1175	1175
q9	5160	4938	4908	4908
q10	6759	2240	1792	1792
q11	471	272	253	253
q12	333	357	209	209
q13	17775	3559	3039	3039
q14	240	226	201	201
q15	516	458	466	458
q16	413	434	372	372
q17	581	852	356	356
q18	7015	6338	6264	6264
q19	1203	968	537	537
q20	317	338	207	207
q21	3000	2171	1963	1963
q22	1061	1024	979	979
Total cold run time: 103360 ms
Total hot run time: 32273 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5480	5415	5494	5415
q2	232	328	241	241
q3	2210	2576	2271	2271
q4	1358	1807	1337	1337
q5	4378	4812	4864	4812
q6	166	170	130	130
q7	2057	2006	1790	1790
q8	2624	2785	2646	2646
q9	7212	7190	7298	7190
q10	3036	3242	2797	2797
q11	574	527	481	481
q12	629	744	579	579
q13	3358	3766	3151	3151
q14	296	293	290	290
q15	508	459	458	458
q16	450	481	425	425
q17	1187	1719	1239	1239
q18	7573	7631	7299	7299
q19	769	1049	1116	1049
q20	1994	2017	1899	1899
q21	5285	5093	4745	4745
q22	1078	1088	980	980
Total cold run time: 52454 ms
Total hot run time: 51224 ms

@freemandealer freemandealer changed the title [enhancement](cloud) Persist LRU information for file cache (pick #49456 #53969) [fix](filecache) unify endian operations and thread renaming method (pick #53140 #53969) Jul 29, 2025
@doris-robot
Copy link

doris-robot commented Jul 29, 2025

TPC-DS: Total hot run time: 196832 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 808b49e691e65ffb536d2457da9aac5a90dac7c6, data reload: false

query1	1309	963	918	918
query2	6240	1923	1872	1872
query3	10978	4325	4446	4325
query4	32983	24121	23546	23546
query5	4163	592	449	449
query6	305	209	194	194
query7	3992	481	322	322
query8	303	245	249	245
query9	9326	2583	2577	2577
query10	475	320	254	254
query11	18010	15813	15420	15420
query12	157	110	104	104
query13	1547	542	427	427
query14	9538	7116	7223	7116
query15	226	208	193	193
query16	7804	660	409	409
query17	1602	803	621	621
query18	2078	423	317	317
query19	217	196	171	171
query20	136	123	131	123
query21	212	140	113	113
query22	4565	4617	4551	4551
query23	34729	34345	34136	34136
query24	7585	2641	2677	2641
query25	486	453	410	410
query26	1170	292	169	169
query27	2132	475	343	343
query28	5546	2226	2168	2168
query29	670	573	431	431
query30	245	195	161	161
query31	1014	914	829	829
query32	69	57	56	56
query33	512	366	299	299
query34	752	852	516	516
query35	760	813	755	755
query36	1025	1041	971	971
query37	108	91	66	66
query38	3969	4005	3951	3951
query39	1554	1474	1456	1456
query40	207	116	106	106
query41	50	51	51	51
query42	123	104	98	98
query43	493	525	484	484
query44	1344	830	835	830
query45	183	178	170	170
query46	878	1050	669	669
query47	2013	1978	1925	1925
query48	416	441	341	341
query49	773	500	436	436
query50	655	712	439	439
query51	7333	7279	7377	7279
query52	103	104	97	97
query53	234	264	199	199
query54	529	570	463	463
query55	79	89	81	81
query56	282	262	254	254
query57	1287	1246	1209	1209
query58	246	224	226	224
query59	3060	3156	2910	2910
query60	287	280	272	272
query61	119	112	121	112
query62	777	776	706	706
query63	223	198	187	187
query64	4048	1018	683	683
query65	3353	3258	3276	3258
query66	1005	421	323	323
query67	16468	15621	15607	15607
query68	5589	835	548	548
query69	501	305	273	273
query70	1166	1135	1124	1124
query71	394	296	281	281
query72	5349	3832	3732	3732
query73	622	739	352	352
query74	10227	9193	8970	8970
query75	3213	3140	2643	2643
query76	3168	1186	756	756
query77	472	349	266	266
query78	10381	10365	9498	9498
query79	3511	857	591	591
query80	755	510	424	424
query81	514	258	228	228
query82	671	124	87	87
query83	157	159	146	146
query84	244	94	83	83
query85	777	370	320	320
query86	379	312	295	295
query87	4309	4325	4148	4148
query88	5199	2388	2379	2379
query89	426	325	291	291
query90	1791	189	189	189
query91	135	137	107	107
query92	68	56	49	49
query93	2489	855	543	543
query94	685	395	280	280
query95	337	281	267	267
query96	480	605	282	282
query97	3187	3285	3162	3162
query98	222	208	199	199
query99	1679	1427	1281	1281
Total cold run time: 294157 ms
Total hot run time: 196832 ms

@doris-robot
Copy link

doris-robot commented Jul 29, 2025

ClickBench: Total hot run time: 29.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 808b49e691e65ffb536d2457da9aac5a90dac7c6, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.07
query4	1.61	0.11	0.10
query5	0.52	0.50	0.51
query6	1.13	0.73	0.72
query7	0.03	0.02	0.01
query8	0.05	0.03	0.03
query9	0.58	0.52	0.52
query10	0.55	0.56	0.58
query11	0.15	0.11	0.11
query12	0.14	0.10	0.11
query13	0.61	0.59	0.59
query14	0.78	0.78	0.79
query15	0.83	0.84	0.83
query16	0.38	0.38	0.40
query17	1.03	1.05	1.04
query18	0.23	0.21	0.21
query19	1.95	1.85	1.78
query20	0.01	0.01	0.01
query21	15.38	0.91	0.56
query22	0.75	0.74	0.71
query23	15.12	1.46	0.54
query24	3.05	1.23	1.98
query25	0.22	0.13	0.08
query26	0.25	0.15	0.14
query27	0.05	0.06	0.06
query28	14.03	0.94	0.43
query29	12.58	3.89	3.23
query30	0.26	0.09	0.07
query31	2.80	0.58	0.38
query32	3.24	0.53	0.45
query33	2.96	2.98	3.02
query34	16.52	5.16	4.59
query35	4.56	4.53	4.57
query36	0.66	0.48	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.14
query41	0.09	0.03	0.02
query42	0.05	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 103.85 s
Total hot run time: 29.14 s

@morrySnow morrySnow changed the title [fix](filecache) unify endian operations and thread renaming method (pick #53140 #53969) branch-3.1: [fix](filecache) unify endian operations and thread renaming method #53140 #53969 Jul 30, 2025
@morrySnow morrySnow merged commit 6953c3d into apache:branch-3.1 Jul 30, 2025
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@freemandealer @hello-stephen @doris-robot @morrySnow