Skip to content

Conversation

@cambyzju
Copy link
Contributor

@cambyzju cambyzju commented Jun 13, 2024

Proposed changes

Issue Number: close #36072

In pr #33630, we already support auto aggregation for random distributed table.

But if all keys are NOT NULL, the aggregation will be eliminated by the rule: ELIMINATE_GROUP_BY

This pr we prevent ELIMINATE_GROUP_BY work for random distributed table.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@cambyzju
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40079 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false

------ Round 1 ----------------------------------
q1	17642	4323	4273	4273
q2	2030	187	188	187
q3	10471	1127	1050	1050
q4	10191	853	770	770
q5	7447	2679	2671	2671
q6	221	137	136	136
q7	967	610	606	606
q8	9219	2056	2029	2029
q9	8921	6461	6462	6461
q10	8963	3698	3700	3698
q11	453	240	231	231
q12	446	224	225	224
q13	17779	2963	2955	2955
q14	275	215	224	215
q15	516	479	475	475
q16	503	376	388	376
q17	960	634	680	634
q18	8364	8104	7759	7759
q19	7176	1401	1430	1401
q20	695	333	355	333
q21	5072	3250	4052	3250
q22	409	345	355	345
Total cold run time: 118720 ms
Total hot run time: 40079 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4577	4377	4437	4377
q2	370	261	270	261
q3	3075	2894	2866	2866
q4	1979	1688	1694	1688
q5	5501	5523	5487	5487
q6	222	129	130	129
q7	2173	1816	1843	1816
q8	3195	3396	3407	3396
q9	8655	8720	8769	8720
q10	4103	3854	3878	3854
q11	605	510	483	483
q12	827	691	630	630
q13	16994	3020	3158	3020
q14	344	258	264	258
q15	521	477	490	477
q16	487	424	445	424
q17	1791	1526	1490	1490
q18	8117	7665	7752	7665
q19	1774	1473	1709	1473
q20	3039	1845	1847	1845
q21	5122	4842	4849	4842
q22	644	532	536	532
Total cold run time: 74115 ms
Total hot run time: 55733 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173333 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false

query1	808	381	383	381
query2	6396	2500	2410	2410
query3	6402	209	210	209
query4	21504	17109	17377	17109
query5	3780	487	470	470
query6	248	167	156	156
query7	4454	311	304	304
query8	338	297	292	292
query9	8373	2453	2456	2453
query10	581	306	274	274
query11	10528	10142	9964	9964
query12	139	83	84	83
query13	1566	365	368	365
query14	8759	6974	6916	6916
query15	244	200	184	184
query16	7239	287	278	278
query17	1397	564	566	564
query18	1913	294	281	281
query19	217	168	169	168
query20	100	91	89	89
query21	206	139	136	136
query22	4566	4352	3994	3994
query23	33590	33068	32986	32986
query24	11109	2877	2808	2808
query25	645	396	379	379
query26	1242	157	157	157
query27	2650	326	336	326
query28	7278	2070	2083	2070
query29	962	681	637	637
query30	274	150	155	150
query31	921	751	746	746
query32	96	57	60	57
query33	774	296	291	291
query34	902	476	475	475
query35	744	626	630	626
query36	1087	918	939	918
query37	164	73	73	73
query38	2868	2732	2737	2732
query39	845	801	813	801
query40	210	139	133	133
query41	56	53	53	53
query42	121	97	101	97
query43	597	581	536	536
query44	1252	751	731	731
query45	199	163	165	163
query46	1075	747	725	725
query47	1868	1754	1801	1754
query48	402	307	308	307
query49	1010	447	426	426
query50	780	403	388	388
query51	6762	6614	6569	6569
query52	104	97	93	93
query53	360	296	293	293
query54	901	475	460	460
query55	76	74	74	74
query56	301	280	286	280
query57	1177	1064	1065	1064
query58	268	262	246	246
query59	3865	3869	3461	3461
query60	316	291	291	291
query61	112	113	109	109
query62	639	443	465	443
query63	321	288	294	288
query64	9068	2408	1851	1851
query65	3259	3095	3076	3076
query66	1117	335	333	333
query67	15572	15007	14874	14874
query68	8396	565	566	565
query69	701	482	546	482
query70	1237	1059	1108	1059
query71	524	274	276	274
query72	8941	5801	5333	5333
query73	1117	333	332	332
query74	5951	5502	5528	5502
query75	4999	2704	2701	2701
query76	4585	930	956	930
query77	811	297	294	294
query78	10581	9752	9646	9646
query79	8523	556	539	539
query80	1098	476	472	472
query81	561	223	221	221
query82	727	107	105	105
query83	343	171	168	168
query84	270	85	86	85
query85	1300	278	318	278
query86	397	323	336	323
query87	3301	3091	3078	3078
query88	5213	2474	2476	2474
query89	554	394	376	376
query90	1989	193	192	192
query91	131	99	100	99
query92	61	51	50	50
query93	7020	519	514	514
query94	1265	194	191	191
query95	403	327	317	317
query96	648	270	270	270
query97	3236	2997	3009	2997
query98	231	202	195	195
query99	1242	872	841	841
Total cold run time: 294053 ms
Total hot run time: 173333 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.50	0.48	0.47
query6	1.12	0.73	0.72
query7	0.01	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.48	0.49
query10	0.53	0.53	0.56
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.76	0.79	0.77
query15	0.86	0.81	0.82
query16	0.37	0.35	0.36
query17	1.00	0.98	1.02
query18	0.21	0.24	0.24
query19	1.75	1.71	1.78
query20	0.02	0.01	0.01
query21	15.42	0.66	0.65
query22	4.01	7.64	1.89
query23	18.22	1.38	1.26
query24	2.14	0.22	0.22
query25	0.15	0.08	0.08
query26	0.27	0.18	0.18
query27	0.08	0.07	0.08
query28	13.22	1.02	0.99
query29	12.57	3.27	3.22
query30	0.27	0.06	0.07
query31	2.84	0.40	0.39
query32	3.26	0.46	0.47
query33	2.83	2.90	2.88
query34	17.00	4.42	4.43
query35	4.44	4.47	4.60
query36	0.65	0.46	0.47
query37	0.20	0.15	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.17	0.15	0.14
query41	0.10	0.04	0.04
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 108.79 s
Total hot run time: 30.42 s

Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 14, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Member

@xy720 xy720 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] SELECT DISTINCT returns duplicate column with aggregate key

6 participants