Skip to content

branch-3.1 [refactor](fe) Merge MasterOpExecutor and FEOpExecutor (#50776) #52061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 3 commits into from
Jun 20, 2025
Merged

branch-3.1 [refactor](fe) Merge MasterOpExecutor and FEOpExecutor (#50776) #52061

morrySnow merged 3 commits into from
Jun 20, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Jun 20, 2025

bp #50776

morningman and others added 3 commits June 20, 2025 16:06
@morningman
[refactor](fe) Merge MasterOpExecutor and FEOpExecutor (apache#50776)
3fd95ad 
Related PR: apache#34685

Problem Summary:

This PR apache#34685 introduced `FEOpExecutor`, which is just a copy of
`MasterOpExecutor`.
Both class are used to forwarding request to a specific FE node, and
there are lots of duplicate code.
So I refactor the code to let `MasterOpExecutor` extends from
`FEOpExecutor`.

No logic being changed in this PR.
@morningman
2
8fdb38f
@morningman
3
4f73ff5
@morningman morningman requested a review from morrySnow as a code owner June 20, 2025 08:08
@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

morningman commented Jun 20, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jun 20, 2025

TPC-H: Total hot run time: 40075 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f73ff514daba41316e924e557c4764dee238d38, data reload: false

------ Round 1 ----------------------------------
q1	17577	6816	6633	6633
q2	2046	164	182	164
q3	10545	1136	1237	1136
q4	10209	789	692	692
q5	7736	2872	2833	2833
q6	213	133	130	130
q7	981	619	602	602
q8	9355	1988	2031	1988
q9	6683	6443	6464	6443
q10	6965	2248	2249	2248
q11	474	263	266	263
q12	413	219	215	215
q13	17790	2953	2946	2946
q14	228	205	207	205
q15	523	478	474	474
q16	503	387	377	377
q17	987	575	570	570
q18	7532	6720	6768	6720
q19	1408	1045	1049	1045
q20	506	199	198	198
q21	3980	3174	3253	3174
q22	1146	1019	1025	1019
Total cold run time: 107800 ms
Total hot run time: 40075 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6607	6916	6545	6545
q2	326	234	233	233
q3	2929	2784	2973	2784
q4	2081	1787	1858	1787
q5	5705	5762	5705	5705
q6	214	124	124	124
q7	2188	1783	1785	1783
q8	3361	3515	3502	3502
q9	8869	8907	8888	8888
q10	3594	3547	3541	3541
q11	583	488	481	481
q12	813	587	614	587
q13	9992	3197	3128	3128
q14	299	258	259	258
q15	519	465	487	465
q16	492	464	434	434
q17	1849	1636	1614	1614
q18	8247	7790	7724	7724
q19	1698	1705	1657	1657
q20	2114	1903	1856	1856
q21	5134	4780	4868	4780
q22	1108	1038	995	995
Total cold run time: 68722 ms
Total hot run time: 58871 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2025

TPC-DS: Total hot run time: 190066 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f73ff514daba41316e924e557c4764dee238d38, data reload: false

query1	999	389	363	363
query2	7317	1908	1943	1908
query3	6716	215	210	210
query4	33891	23873	23357	23357
query5	4358	449	435	435
query6	270	174	181	174
query7	4617	307	315	307
query8	307	242	222	222
query9	9633	2590	2598	2590
query10	486	260	259	259
query11	18463	15123	15217	15123
query12	156	111	99	99
query13	1643	430	409	409
query14	10248	6829	7015	6829
query15	248	171	182	171
query16	8115	479	476	476
query17	1633	582	571	571
query18	2132	308	318	308
query19	307	172	157	157
query20	121	111	110	110
query21	208	103	106	103
query22	4446	4385	4082	4082
query23	34397	33531	33735	33531
query24	11324	2857	2833	2833
query25	680	408	414	408
query26	1508	167	168	167
query27	2925	348	352	348
query28	7771	2121	2121	2121
query29	964	448	450	448
query30	318	169	165	165
query31	1027	767	798	767
query32	92	59	58	58
query33	789	312	311	311
query34	926	503	497	497
query35	836	746	710	710
query36	1138	938	950	938
query37	138	70	72	70
query38	3915	3811	3816	3811
query39	1509	1421	1411	1411
query40	291	105	105	105
query41	54	51	52	51
query42	116	109	108	108
query43	531	492	488	488
query44	1201	811	813	811
query45	184	171	172	171
query46	1127	722	704	704
query47	1914	1830	1828	1828
query48	471	394	386	386
query49	1210	420	408	408
query50	807	428	406	406
query51	7242	6989	7201	6989
query52	107	92	96	92
query53	255	188	186	186
query54	1183	468	483	468
query55	82	80	79	79
query56	277	262	262	262
query57	1268	1184	1175	1175
query58	251	236	220	220
query59	3105	2944	2817	2817
query60	283	253	253	253
query61	115	111	115	111
query62	851	668	656	656
query63	220	188	189	188
query64	5329	662	642	642
query65	3274	3206	3184	3184
query66	1337	309	326	309
query67	15918	15465	15495	15465
query68	4826	577	609	577
query69	424	270	253	253
query70	1160	1118	1107	1107
query71	376	256	248	248
query72	6439	4219	4068	4068
query73	759	346	349	346
query74	10129	9290	8988	8988
query75	3413	2671	2665	2665
query76	3046	1057	1091	1057
query77	402	277	272	272
query78	10549	9773	9579	9579
query79	2393	595	614	595
query80	1012	444	419	419
query81	534	221	222	221
query82	947	94	89	89
query83	226	153	145	145
query84	230	79	82	79
query85	1297	328	306	306
query86	427	303	310	303
query87	4373	4263	4283	4263
query88	4012	2359	2348	2348
query89	407	304	298	298
query90	1872	184	182	182
query91	187	150	149	149
query92	83	51	49	49
query93	1626	566	562	562
query94	764	308	300	300
query95	350	254	256	254
query96	610	283	280	280
query97	3281	3209	3133	3133
query98	213	201	191	191
query99	1517	1321	1321	1321
Total cold run time: 303458 ms
Total hot run time: 190066 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2025

ClickBench: Total hot run time: 30.13 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4f73ff514daba41316e924e557c4764dee238d38, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.02	0.03
query3	0.24	0.06	0.07
query4	1.62	0.10	0.11
query5	0.53	0.50	0.50
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.55	0.54	0.56
query11	0.14	0.10	0.12
query12	0.14	0.11	0.11
query13	0.62	0.59	0.59
query14	0.78	0.79	0.80
query15	0.84	0.82	0.82
query16	0.40	0.36	0.38
query17	1.06	1.06	1.07
query18	0.23	0.22	0.22
query19	1.88	1.82	1.81
query20	0.02	0.01	0.01
query21	15.41	0.59	0.58
query22	2.81	2.29	1.39
query23	17.15	0.94	0.89
query24	2.94	1.26	2.28
query25	0.23	0.05	0.05
query26	0.68	0.13	0.13
query27	0.04	0.03	0.04
query28	9.13	0.52	0.49
query29	12.58	3.19	3.19
query30	0.24	0.06	0.06
query31	2.87	0.38	0.40
query32	3.26	0.46	0.45
query33	3.01	2.98	3.06
query34	16.99	4.56	4.61
query35	4.57	4.58	4.55
query36	0.66	0.48	0.46
query37	0.08	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.94 s
Total hot run time: 30.13 s

@morrySnow morrySnow merged commit cd3683b into apache:branch-3.1 Jun 20, 2025
21 of 22 checks passed
@morningman morningman mentioned this pull request Aug 14, 2025
Merged
morrySnow pushed a commit that referenced this pull request Aug 14, 2025
@morningman
branch-3.1: [fix](forward) fix cherry-pick error in 3.1 (#54740)
4f2a280 
introduced from #52061
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@morningman @hello-stephen @doris-robot @morrySnow