Skip to content

branch-3.1: [fix](local shuffle) add distribute exprs to set operation node #56157 #56180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 28, 2025
Merged

branch-3.1: [fix](local shuffle) add distribute exprs to set operation node #56157 #56180

morrySnow merged 1 commit into from
Sep 28, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 18, 2025

Cherry-picked from #56157

@github-actions github-actions bot requested a review from morrySnow as a code owner September 18, 2025 04:18
@Thearas
Copy link
Contributor

Thearas commented Sep 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Sep 18, 2025
@Thearas
Copy link
Contributor

Thearas commented Sep 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 18, 2025

TPC-H: Total hot run time: 32930 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b18a568262a189dd71c951862c9d03b8350716d0, data reload: false

------ Round 1 ----------------------------------
q1	18017	5494	5468	5468
q2	2022	396	282	282
q3	11887	1306	779	779
q4	10379	885	466	466
q5	9628	2386	2163	2163
q6	187	166	139	139
q7	920	754	618	618
q8	9346	1451	1190	1190
q9	5226	4923	4942	4923
q10	6770	2262	1824	1824
q11	473	279	254	254
q12	337	354	213	213
q13	17782	3604	3036	3036
q14	215	227	203	203
q15	523	471	460	460
q16	419	424	385	385
q17	601	874	373	373
q18	6828	6476	6290	6290
q19	1216	955	579	579
q20	343	345	214	214
q21	3141	2282	2090	2090
q22	1045	1014	981	981
Total cold run time: 107305 ms
Total hot run time: 32930 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5566	5497	5528	5497
q2	232	326	235	235
q3	2231	2611	2321	2321
q4	1353	1866	1432	1432
q5	4427	5036	5003	5003
q6	170	160	132	132
q7	2099	1986	1824	1824
q8	2640	2824	2716	2716
q9	7275	7271	7179	7179
q10	3003	3210	2750	2750
q11	573	530	489	489
q12	681	740	615	615
q13	3392	3798	3157	3157
q14	280	300	276	276
q15	520	471	465	465
q16	437	473	433	433
q17	1233	1746	1269	1269
q18	7697	7439	7266	7266
q19	832	1130	1160	1130
q20	2028	2042	1879	1879
q21	5520	4897	4561	4561
q22	1080	1076	1032	1032
Total cold run time: 53269 ms
Total hot run time: 51661 ms

@doris-robot
Copy link

doris-robot commented Sep 18, 2025

TPC-DS: Total hot run time: 191986 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b18a568262a189dd71c951862c9d03b8350716d0, data reload: false

query1	949	388	393	388
query2	6211	1924	1898	1898
query3	8681	207	203	203
query4	33801	23612	23190	23190
query5	3731	587	454	454
query6	295	206	199	199
query7	4205	509	323	323
query8	296	241	230	230
query9	9423	2588	2576	2576
query10	472	327	258	258
query11	18395	15417	15205	15205
query12	155	106	105	105
query13	1555	556	425	425
query14	9042	6659	7662	6659
query15	238	192	188	188
query16	8080	687	526	526
query17	1568	780	601	601
query18	2117	425	334	334
query19	229	191	166	166
query20	126	129	120	120
query21	210	139	111	111
query22	4600	4595	4449	4449
query23	34472	33948	34250	33948
query24	7393	2760	2751	2751
query25	545	489	427	427
query26	820	289	179	179
query27	2205	492	373	373
query28	5351	2274	2244	2244
query29	736	604	471	471
query30	242	190	178	178
query31	994	930	849	849
query32	94	62	62	62
query33	502	387	331	331
query34	768	865	527	527
query35	796	796	724	724
query36	1038	1100	959	959
query37	110	93	71	71
query38	4068	4045	3946	3946
query39	1558	1464	1491	1464
query40	204	123	119	119
query41	50	50	45	45
query42	119	100	103	100
query43	513	524	504	504
query44	1380	856	843	843
query45	187	177	171	171
query46	889	1074	691	691
query47	1930	1971	1922	1922
query48	436	435	357	357
query49	738	501	394	394
query50	698	701	445	445
query51	7266	7377	7219	7219
query52	104	102	103	102
query53	242	273	205	205
query54	570	559	478	478
query55	80	85	87	85
query56	282	266	273	266
query57	1267	1239	1228	1228
query58	247	228	225	225
query59	3041	3204	3031	3031
query60	294	285	266	266
query61	114	118	122	118
query62	819	768	682	682
query63	240	195	194	194
query64	4010	1014	639	639
query65	3390	3366	3333	3333
query66	785	405	307	307
query67	16397	15783	15482	15482
query68	7690	850	560	560
query69	506	308	270	270
query70	1226	1112	1086	1086
query71	371	294	269	269
query72	5803	3750	3821	3750
query73	664	743	357	357
query74	10726	9156	9262	9156
query75	3169	3177	2678	2678
query76	2954	1170	795	795
query77	497	370	282	282
query78	10353	10534	9687	9687
query79	3641	861	594	594
query80	681	508	438	438
query81	522	249	220	220
query82	596	117	89	89
query83	170	157	154	154
query84	238	106	81	81
query85	786	397	302	302
query86	404	298	290	290
query87	4314	4273	4208	4208
query88	5244	2437	2426	2426
query89	410	340	296	296
query90	1738	188	188	188
query91	135	148	111	111
query92	68	53	49	49
query93	2108	887	568	568
query94	698	409	309	309
query95	337	286	271	271
query96	500	611	280	280
query97	3213	3266	3169	3169
query98	240	206	197	197
query99	1505	1415	1293	1293
Total cold run time: 293870 ms
Total hot run time: 191986 ms

@doris-robot
Copy link

doris-robot commented Sep 18, 2025

ClickBench: Total hot run time: 28.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b18a568262a189dd71c951862c9d03b8350716d0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.51	0.52	0.51
query6	1.13	0.74	0.72
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.58	0.51	0.50
query10	0.56	0.56	0.55
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.62	0.61	0.60
query14	0.77	0.81	0.81
query15	0.84	0.82	0.83
query16	0.37	0.41	0.41
query17	1.02	1.03	1.05
query18	0.25	0.22	0.22
query19	1.96	1.82	1.73
query20	0.02	0.02	0.01
query21	15.39	0.91	0.59
query22	0.74	0.91	0.66
query23	15.02	1.48	0.55
query24	3.05	0.52	1.71
query25	0.12	0.21	0.09
query26	0.34	0.16	0.14
query27	0.04	0.05	0.07
query28	13.63	1.01	0.42
query29	12.59	3.86	3.25
query30	0.24	0.09	0.06
query31	2.85	0.60	0.38
query32	3.22	0.53	0.47
query33	3.01	3.02	3.04
query34	16.90	5.16	4.54
query35	4.59	4.60	4.55
query36	0.64	0.50	0.47
query37	0.09	0.05	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.17	0.14	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.8 s
Total hot run time: 28.44 s

@morrySnow morrySnow merged commit 5345313 into branch-3.1 Sep 28, 2025
23 checks passed
@morrySnow morrySnow deleted the auto-pick-56157-branch-3.1 branch September 28, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @morrySnow @dataroaring